Teadmiste otsing, formaliseerimine ja hoidmine: erinevus redaktsioonide vahel

Allikas: Lambda
Mine navigeerimisribale Mine otsikasti
← Vanem muudatus
Kalle (arutelu | kaastöö)
9 muudatust
Resümee puudub
 
(ei näidata 2 kasutaja 267 vahepealset redaktsiooni)
8. rida: 8. rida:
Link: http://www.lambda.ee/index.php/Teadmiste_otsing,_formaliseerimine_ja_hoidmine  or http://www.lambda.ee/index/itv0060<br>
Link: http://www.lambda.ee/index.php/Teadmiste_otsing,_formaliseerimine_ja_hoidmine  or http://www.lambda.ee/index/itv0060<br>
Lecturer: Tanel Tammet<br>
Lecturer: Tanel Tammet<br>
Contact: tammet@staff.ttu.ee, 6203457, TTÜ AK223<br>
Contact: tanel.tammet@ttu.ee, 6203457, TTÜ ICT-426<br>
Archives of previous years: [[itv0060 2011 arhiiv|2011]], [[itv0060 20010 arhiiv|2010]], [[itv0060 2009 arhiiv|2009]], [[itv0060 2008 arhiiv|2008]], [[itv0060 2007 arhiiv|2007]],  [[itv0060 2006 arhiiv|2006]], [http:/cs.ttu.ee/kursused/wav4130/ older].
Archives of previous years: [[itv0060 2014 arhiiv|2014]], [[itv0060 2013 arhiiv|2013]], [[itv0060 2012 arhiiv|2012]], [[itv0060 2011 arhiiv|2011]], [[itv0060 20010 arhiiv|2010]], [[itv0060 2009 arhiiv|2009]], [[itv0060 2008 arhiiv|2008]], [[itv0060 2007 arhiiv|2007]],  [[itv0060 2006 arhiiv|2006]], [http://cs.ttu.ee/kursused/wav4130/ older].
</td>
</td>
</tr>
</tr>
</table>
</table>


== <font color="red">NB! This is an archive from 2015</font> ==
== Exam ==
Date and time (two separate occasions):
* 2. June (Tuesday) at 10:00 in room <b>SOC-214</b> (Majandus- ja sotsiaalteaduste maja)
* 9. June (Tuesday) at 10:00 in room <b>SOC-210</b> (Majandus- ja sotsiaalteaduste maja)
4 questions, one for each block. Each question is a small task in encoding information. No programming excersices.
* 1. task: encoding data in a conventional relational database in RDF, inventing suitable ID-s, converting selects with where conditions and joins to first order rule form.
* 2. task: encoding data/rules in RDFs (perhaps additionally in RDFa). Writing 1st order rules about the meaning of RDFs built-in axioms and deriving data using RDFs built-in axioms. Concrete derivation systems are not required: just deriving correct information. It is also important to understand ways to uri-s as global id-s and encoding data in both html metainfo part and in pages, using RDFa.
* 3. task: encoding uncertain knowledge. You will first have to encode a small scenario using several different systems: default logic, probabilistic logic, fuzzy logic. Second, you have to show what can you derive using one or another formalism.
* 4. tasks: building indexes. Given a small dataset(s), you will have to build several indexes. Which indexes: trie indexes for words in a text, discrimination tree indexes for generalizations of terms, path indexes for concretisations of terms. For the latter two, use the [http://www.lambda.ee/index.php/Pilt:Mccuneindexing.pdf mccune paper]
NB! The information in the following "this is in exam ..." is old and not relevant.


== Time, place, result ==
== Time, place, result ==
19. rida: 40. rida:
Semester: spring<br>
Semester: spring<br>
Grading: exam<br>
Grading: exam<br>
Points: 3.5<br>


Lectures: every Monday 12.00-13.30, room IT-137a<br>
Lectures: every Monday 16:00-17.30, room ICT-A1<br>
Practical work: Mondays on odd weeks 10.00-11.30, room IT-213H:<br>31 jan, 14 feb, 28 feb, 14 mar, 28 mar, 4 apr, 18 apr, 2 may, 16 may.
Practical work: Mondays on odd weeks (2. Feb, 16 Feb, ...) 17:45-19:15 room ICT-403<br>
Practical work will give 40% and exam 60% of points underlying the final grade.
The exam will consist of several small excercises.


Practical work will give 40% and exam 60% of points underlying the final grade.
== Focus ==


The main focus of the course is on KR ([https://en.wikipedia.org/wiki/Knowledge_representation knowledge representation]): how to represent nontrivial information in programs and databases, how to build and use indexes for efficient search through large sets of knowledge. Check out [http://www.lisperati.com/tellstuff/ this description].


== <font color="red">Järeleksam 2011 sügis</font> ==
The course contains four blocks built on each other:


Järeleksam toimub 26 august (reede): kell 10 auditoorium IT-137a
* Background and basics. Representing simple facts.
* Representing rules.
* Time, planning and uncertain knowledge.
* Indexes and search.


== Eksam ja konsultatsioon ==
Books to use:


16. mai: eksamiks õppimise materjalide ülevaade ja fikseerimine.
* Ca half of the course themes are covered in [http://www.sciencedirect.com/science/book/9781558609327 this book] with freely accessible pdf-s.
* The freely accesible pdf of the [http://ii.fmph.uniba.sk/kri/KRhandbook.pdf handbook if knowledge representation] gives a detailed and thorough coverage of the subject; far more than necessary for the course.
* Interesting to browse: [http://www.aaai.org/Library/KR/kr12contents.php recent conference proceedings]
* Interesting to browse: [http://www.cse.buffalo.edu/~shapiro/Courses/CSE563/Slides/krrSlides.pdf course materials], [http://web.stanford.edu/class/cs227/ course materials]


Eksam:
Observe that a noticeable part of the course contents are not covered by these books: use the course materials and links to papers, standards and tutorials provided.


== Practical work ==


järeleksam 15 juuni (kolmapäev): kell 9 auditoorium IT-137a
There are three labs: the first two are obligatory, third is optional. The labs have to be presented to the prof and all students present at labwork.


<font color="red">15 juuni tulemused:</font>, eksamipunktid viidud 0-60 skaalale:
=== First lab 2015 ===
* HP eksam 52p + 40p ettekanded, hinne 5
* GJ eksam 31p + 32p ettekanded, hinne 2


3 juuni (reede): kell 9 auditoorium VI-121
The goal of the first lab is to write a software system able to scrape factual raw data about an address (like "Akadeemia tee 12, Tallinn") given to the program.


<font color="red">3 juuni tulemused:</font>, eksamipunktid viidud 0-60 skaalale:
Basically, search for web pages containing the name or similar names and extract relevant words from the list you create: words closer to the name and with more occurrences are also more important/better match. Titles/headlines above the word are also more likely to be important. It may be a good idea to do some searches together with interesting keywords already. Also, whenever you do a search / pull a page, it is a good idea to store the search result / html source in a file to avoid exchausting your search quota and just wasting bandwidth and time,
* TT eksam 36p + 40p ettekanded, hinne 3
* RL eksam 40p + 40p ettekanded, hinne 4
* HP eksam 28p + 40p ettekanded, hinne 2
* VP eksam 22p + 40p ettekanded, hinne 2
* DA eksam 57p + 40p ettekanded, hinne 5


8 juuni (kolmapäev): kell 9 auditoorium VII-131
It makes sense to search adresses in several ways: googling, binging plus searching from known real estate portals.


<font color="red">8 juuni tulemused:</font>, eksamipunktid viidud 0-60 skaalale:
Deadline: <b>16. March</b> (after this there will be a penalty).
* KL eksam 60p + 40p ettekanded, hinne 5
* ML eksam 51p + 40p ettekanded, hinne 5
* ET eksam 48p + 30p ettekanded, hinne 3
* DF eksam 58p + 40p ettekanded, hinne 5


Järeleksam kevadel:
See also [[notes for KR lab 1]].


* 15 juuni (kolmapäev): kell 9 auditoorium IT-137a
=== Second lab 2015 ===


== Practical work ==
The goal of the second lab is to write and use rules to categorize/tag addresses according to the data obtained during the first lab. An address should get a number of tags with numerical indicators showing our trust in that the tag really applies to the address, plus (sometimes) a number indicating the degree to which the tag applies.


There are two labs on the same theme and an optional third lab.


=== First lab ===
See [[details for KR lab 2]].


The main goal of the first lab is to web-scrape data from the public sources using search engines and simple text analysis/statistics. 
=== Third lab 2014 ===


Details in [[Teadmised first lab 2011|Teadmised first lab]]. The lab should be completed by March, later presentation will cut the points to half.
This lab is <b>optional</b> and will simply give as many points as lab 1 or 2 towards the final result and the grade: practical wotrk 60% and exam 40%.


=== Second lab ===
The goal of the lab is to use [http://wordnet.princeton.edu/ wordnet] or [http://www.cl.ut.ee/ressursid/teksaurus/ teksaurus] to create an additional ruleset and to use that
in addition to your own rules in lab 2.


The main goal of the second lab is to augment data obtained from the first lab by writing and using probabilistic rules with a rule engine.
=== Lab ideas ===


Details in [[Teadmised second lab 2011]].
* [[KR lab idea by Priit Järv]]


=== Optional third lab ===
Student ideas are also welcome: have to agree with Tanel first.


The optional third lab can be done in parallel with the second. It will be a relatively complex lab: successful passing will give additional 20 points, thus the weight of the exam is diminished from 60% to 40% of the overall score.  
A cool idea is to investigate:
* http://threatpost.com/fbstalker-automates-facebook-graph-search-data-mining
* https://github.com/milo2012/osintstalker


What to do: use Attempto to create a system such that you can write rules and/or facts and/or queries for the second lab in restricted english. These restricted english sentences have to be automatically converted to the form usable by Otter. It is a good idea to avoid probabilities in these rules, since it would be very hard to encode probabilities using attempto.




See: http://attempto.ifi.uzh.ch/site/ , http://attempto.ifi.uzh.ch/ape/


== Materials to study for the exam ==
== Lecture block 1: basics and representing simple facts. ==


Eksamiküsimused:


Väikesed ülesanded: data ja reeglite
=== Lecture 1 (2. Feb) Overview of the course. Background and basics. First lab.  ===
esitamine ja kasutamine


Andmebaas nö tüüpilise relatsioonilise
Lecture materials:
mudelina, key-value ja rdf kujul
* [[Media:Kr intro.ppt | Intro lecture: declarative and procedural representations]]
* Core logic refresher.
* Details of the first lab.


Ülesanne blocks worldist: esita mingi
=== Lecture 2 (9 Feb) Programming and databases. SQL: meaning and representation of facts ===
olukord, mingi reegel ja näita rakendamise
tulemust


RDF, RDFS ja OWL põhiasjad: väike
* Encoding data in programming languages.
ülesanne. Süntaksid ja eridetailid
* [[Media:Kr_plain_data_db_logic.ppt | relation of plain data in databases to logic & representing complex structures in databases ]]
ei ole olulised.
* Core ideas of non-relational databases, mostly RDF


RDF ja RDFS esitada loogikas
Lecture materials:
faktide ja reeglitena, mõni
lihtne OWLi näide võib ka olla


Arusaamine RDFa põhiasjadest, oskus
* Start with [[Media:kr_rdf_schemalessdb_3_2.ppt|key/value pairs and rdf, compared to the relational model]]
teha näidet.
* [[Nosql links and notes]]


MÕni üldküsimus Nelli ja Attempto
=== Lecture 3 (16 Feb) HTML annotations. Microformats, microdata, RDFa  ===
kohta.


Arusaam et mis on mittemonotoonne
Two student presentations ca 30 min each:
loogika ja mis ei ole, mis oleks
viisid esitada vasturääkivaid
teadmisi. Võib olla väike ülesanne
mingite asjade esitamise kohta: näide
ja selgitus.


Default loogika ülesanne a la linnud
* Microformats, microdata, RDFa: Madis Taimre
ja Quaker diamond: panna kirja reeglid
* Facebook open graph: Madis Allikmaa
ja viia läbi järeldused ja vbl selgitada
lisaks. Seos CWA, Prolog ja default loogika
vahel võib samuti küsimuse alla tulla.


Indeksid: arusaam baasindekseerimis-ideedest
Lecture materials:
ja B+ treest. Ntx ehitada B+ mingi
väikese datahulga jaoks. Sama hashi kohta
ja bit indeksite kohta. Eriteemad a la
indeksi pakkimine ja erinevad hashi
viisid ei tule teemaks.


Täistekstiindeksid: osata ehitada näite
* [[Media:Kr_html_annotations.ppt|Html annotations]]
pealt. Vt presentatsioonist erinevaid
viise, ntx Patricia trie.


Termiindeksid: piisab discrimination
Understand main parts:
tree indeksist ja näite põhjal
indeksi ehitamisest.


* [https://support.google.com/webmasters/answer/99170?hl=en Google rich snippets]
* [http://en.wikipedia.org/wiki/RDFa wiki intro]
* [http://www.w3.org/TR/rdfa-lite/ RDFa lite] ja [http://www.w3.org/TR/xhtml-rdfa-primer/ w3c rdfa primer]
* [http://ogp.me/ open graph protocol (i.e. Facebook stuff)]
* [http://microformats.org/ microformats]
* [[Media:portaalidekoosvoime.ppt]] or as pdf [[Media:portaalidekoosvoime.pdf]]


=== Declarative and procedural representations ===
=== Lecture 4 (2. March) RDF last parts + RDFS start ===


Continuing with RDF:


Lecture material:
* Continue with the lecture material [[Media:kr_rdf_schemalessdb_3_2.ppt|key/value pairs and rdf, compared to the relational model]]
* [[Media:Kr intro.ppt | Intro lecture: declarative and procedural representations]]
* Check out http://www.w3.org/RDF/ and http://www.w3.org/standards/techs/rdf#w3c_all
* And read more from http://www.w3.org/TR/rdf11-primer/ and http://www.w3.org/TR/rdf11-new/


=== Databases: meaning and representation of facts ===
Starting RDFS:


* [[Media:Kr_plain_data_db_logic.ppt | relation of plain data in databases to logic & representing complex structures in databases ]]
* Start with the lecture material [[Media:Kr_rdfs.pdf|RDFS: rdf schema]] and as [[Media:Kr_rdfs.ppt|ppt]]
* [[Media:Kr_rdf_schemalessdb_3.pdf|key/value pairs and rdf, compared to the relational model]]  
* Read [http://en.wikipedia.org/wiki/RDF_schema RDFs wikipedia]
* Check out [http://www.w3.org/TR/rdf-schema/ w3c RDF schema]


=== Rules in logic: apps in planning, robotics, ... ===
Choosing the student presentations from:  
* [[Nosql links and notes]]


Lecture material:
== Lecture block 2: representing rules ==
* [[Media:Kr_rules1.pdf|Rules and logic: refresher]] and as [[Media:Kr_rules1.ppt|ppt]]


See also:


* [[Media:Geoffreasoningnotes.pdf|Geoff reasoning course notes: just for background, no questions in exam]]
=== Lecture 5 (9. March) Student presentations on NELL and nosql databases + intro to the second lab ===
* [http://www.ee.pdx.edu/~mperkows/May4/005.Ch11-planning.ppt planning and blocks world]
* [[blocks world axiom/query examples from lecture]]


Student lectures on the theme of data and knowledge extraction from the web:
* Nell and systems like Nell (are there any? Check also ConceptNet): Andrei. See some details about [http://rtw.ml.cmu.edu/rtw/ Nell] as a nice example.
* Student presentations chosen from [[Nosql links and notes]]:
** Rait: docs.mongodb.org/manual/  (tehtud)
** Raigo: http://www.neo4j.org/ (tehtud)


=== RDF, RDFS, OWL and conversion to 1st order logic ===
Understand RDFS:


Lecture material:
Lecture material:
* [[Media:Kr_rdfs.pdf|RDFS: rdf schema]] and as [[Media:Kr_rdfs.ppt|ppt]]


Understand RDF and RDFS:
* Continue with [[Media:Kr_rdfs.pdf|RDFS: rdf schema]] and as [[Media:Kr_rdfs.ppt|ppt]]


* [http://en.wikipedia.org/wiki/Resource_Description_Framework RDF wikipedia]
Additional details (not part of exam):
* [http://www.w3.org/TR/rdf-primer/ w3c RDF primer]


Understand basics of owl:
* [http://en.wikipedia.org/wiki/RDF_schema RDFs wikipedia]
* [http://www.w3.org/TR/rdf-schema/ w3c RDF schema]


* [http://en.wikipedia.org/wiki/Web_Ontology_Language OWL wikipedia]
=== Lecture 6 (16. March) continuing RDFs and looking into other KR languages ===
* [http://www.w3.org/TR/owl-guide/ w3c OWL guide]


=== RDFa, reading and using RDF data from web pages ===


Understand main parts:
Lecture material:


* [http://en.wikipedia.org/wiki/RDFa wiki intro]
* Continue with [[Media:Kr_rdfs.pdf|RDFS: rdf schema]] and as [[Media:Kr_rdfs.ppt|ppt]]
* [[Media:portaalidekoosvoime.ppt]] or as pdf [[Media:portaalidekoosvoime.pdf]]  
* [http://www.w3.org/2006/07/SWD/RDFa/primer/ w3c rdfa primer]


=== RDF relation to logic ===
Additional details (not part of exam):


Notes from lecture:
* [http://en.wikipedia.org/wiki/RDF_schema RDFs wikipedia]
* [[Notes about RDF and OWL: logical meaning]]
* [http://www.w3.org/TR/rdf-schema/ w3c RDF schema]


Understand a bit of:
* [http://www.w3.org/TR/2003/NOTE-lbase-20031010/ Hayes & Guha Lbase]


=== Semantics of ordinary relational base ===
Important KR languages:


* [http://cs.ttu.ee/kursused/wav4130/rdftutapp.pdf rdf tutorial addendum for logic and dbases]
* RuleML [http://wiki.ruleml.org/index.php/RuleML_Home RuleML ]
 
* OWL [http://en.wikipedia.org/wiki/Web_Ontology_Language OWL wikipedia],  [http://www.w3.org/TR/owl-guide/ w3c OWL guide]
===  First order logic in query engines and planning ===
* KIF
* CL [http://iso-commonlogic.org CL main page],
* ontologies
* wordnet
* cyc
* TPTP language [http://www.cs.miami.edu/~tptp/TPTP/QuickGuide/ TPTP]
<!-- * restricted english systems [http://attempto.ifi.unizh.ch/site/pubs/papers/phd_kaljurand.pdf Attempto restricted english] -->


* [[Media:Geoffreasoningnotes.pdf|Geoff reasoning course notes: just for background, no questions in exam]]
=== Lecture 7 (23. March): OWL and ontologies ===
* [http://www.ee.pdx.edu/~mperkows/May4/005.Ch11-planning.ppt planning and blocks world]
* [[blocks world axiom/query examples from lecture]]


Owl background: description logics (not part of exam )


=== Data extraction ===
* [https://www.inf.unibz.it/~franconi/dl/course/slides/prop-DL/propositional-dl.pdf description logic intro]
* [http://disi.unitn.it/~ldkr/ml2013/slides/12.DL.ppt brief alternative intro to description logic]
* [http://www.inf.unibz.it/~franconi/dl/course/ detailed course in description logics] (not necessary for exam)


Web mining by Nell:
Understand basics of owl:
* [http://rtw.ml.cmu.edu/rtw/ Nell]


===  Restricted english jms ===
* [http://en.wikipedia.org/wiki/Web_Ontology_Language OWL wikipedia]


* [http://attempto.ifi.unizh.ch/site/pubs/papers/phd_kaljurand.pdf Attempto restricted english]
not part of exam:


=== Uncertain knowledge logic ===
* [http://www.w3.org/TR/owl2-overview/ w3c OWL version 2 guide]
* [http://www.w3.org/TR/owl2-primer/ w3c OWL version 2 primer]
* [http://www.w3.org/2007/OWL/wiki/Profiles#Reasoning_in_OWL_2_RL_and_RDF_Graphs_using_Rules Owl2 wiki rule semantics] 


* [http://plato.stanford.edu/entries/logic-nonmonotonic/ nonmonotonic logic]
[[Notes about RDF and OWL: logical meaning]]
* [http://en.wikipedia.org/wiki/Default_logic default logic]
* [http://sc.snu.ac.kr/courses/2008/spring/pg/krr/Lectures/Lecture11/Lec11.ppt Reasoning with Conflicting Knowledge]


=== Indexes ===
Start looking at interesting ontologies:


Traditional database indexes incl B+ tree:
* [http://wordnet.princeton.edu/ Wordnet]
* [http://www.lambda.ee/index.php/Pilt:IndexNotes04.ppt G. Molina lecture 4]
* [http://www.cl.ut.ee/ressursid/teksaurus/ Teksaurus]
* [http://conceptnet5.media.mit.edu/ Conceptnet]
* [https://schema.org/docs/full.html schema.org]
* [http://www.ontologyportal.org/ SUMO]


Bitmap indexes:
===  Lecture 8 (30 March): Restricted english ===
* [http://en.wikipedia.org/wiki/Bitmap_index wiki]


Hash indexes:
* [http://attempto.ifi.uzh.ch/site/pubs/ Attempto restricted english] as intro use [http://attempto.ifi.uzh.ch/site/talks/files/Talk.Stanford.05.pdf this talk]
* [http://en.wikipedia.org/wiki/Hash_table wiki]
* [http://inform7.com/ Inform] system for interactive fiction, vt ka [http://inform-fiction.org/manual/html/contents.html vanemat raamatut]


Fulltext indexes:
Attempto details not necessary for the exam.
* [http://www.lambda.ee/index.php/Pilt:Fulltext_indexes.ppt good intro and overview]


Term indexes:
== Lecture block 3: time, planning and uncertain knowledge ==
Understand basics of discrimination tree trie-type term indexes and path indexes from
* [http://www.lambda.ee/index.php/Pilt:Mccuneindexing.pdf mccune paper]


== Course structure ==


We start with representing and converting facts, continue to rules, first monotonic, then probabilistic and nonmonotonic, then look into search and indexing of complex information, and finally, some learning.
=== Rules in planning and robotics ===


=== Intro and paradigms of knowledge representation ===
Lecture material:
* [[Media:Kr_rules1.pdf|Rules and logic: refresher]] and as [[Media:Kr_rules1.ppt|ppt]]
* [[blocks world axiom/query examples from lecture]]


Lecture materials:
See also (not part of exam):
* [[Media:Kr intro.ppt | Intro lecture: declarative and procedural representations]]
* [http://dl.dropbox.com/u/11794296/roboswarm_saso_2.pdf roboswarm example of combining dec and proc]


For KR, declarative and procedural rep see:
* [[Media:Geoffreasoningnotes.pdf|Geoff reasoning course notes: just for background, no questions in exam]]
* [http://en.wikipedia.org/wiki/Knowledge_representation Short wiki overview of knowledge representation]
* [http://www.ee.pdx.edu/~mperkows/May4/005.Ch11-planning.ppt planning and blocks world]
* [http://groups.csail.mit.edu/medg/ftp/psz/k-rep.html Long and philosophical MIT overview of knowledge representation]
* [http://www.cse.buffalo.edu/~rapaport/676/F01/proc.decl.html Pointers to classical papers]
* [http://www.cogarch.org/index.php/Main_Page A Survey of Cognitive and Agent Architectures]
* [ftp://reports.stanford.edu/pub/cstr/reports/cs/tr/76/581/CS-TR-76-581.pdf Classical KRL paper from 1977: Bobrow & Winograd]
Web mining by Nell:
* [http://rtw.ml.cmu.edu/rtw/ Nell]
* [http://www.cs.cmu.edu/~rcwang/ Richard Wang]
* [http://www.boowa.com/ boowa extractor]


=== Databases: meaning and representation of facts ===
=== Logic for uncertain knowledge ===


Lecture material:
* [http://plato.stanford.edu/entries/logic-nonmonotonic/ nonmonotonic logic] Not necessary for exam.
* [http://en.wikipedia.org/wiki/Default_logic default logic] For exam: main material for default logic.
<!--
* [http://sc.snu.ac.kr/courses/2008/spring/pg/krr/Lectures/Lecture11/Lec11.ppt Reasoning with Conflicting Knowledge] Helpful additional material.
-->


<!-- * [[Media:Kr_plain_data_db_logic_draft.ppt|logic and databases intro draft]] -->
For the exam: you should be able to create and solve small examples with default logic.
* [[Media:Kr_plain_data_db_logic.ppt | relation of plain data in databases to logic & representing complex structures in databases ]]
<!-- * [[Media:Kr_prop_val_pairs.ppt|property/value pairs]] -->
* [[Media:Kr_rdf_schemalessdb_3.pdf|key/value pairs and rdf, compared to the relational model]]


See also about databases and logic:
=== Fuzzy and probabilistic logic ===


* [http://www.cs.rice.edu/~tlogic/Database/all-lectures.pdf Logic and Database Queries (readable)]
* [[Media:Uncertain_prob_fuzzy.ppt‎|Uncertain_prob_fuzzy.ppt‎]] Intro.
* [http://magenta.ruc.dk/dat_en/research/reports/127.pdf logic in databases workshop (complex stuff)]
* [[Media:Vienna_tanel.pdf|Vienna_tanel_2.pdf]] Additional examples and combining.


See also about RDF and related:
For the exam: understand the differences between fuzzy and probabilistic logic and be able to present small examples.


* [http://cs.ttu.ee/kursused/wav4130/rdftutapp.pdf rdf tutorial addendum for logic and dbases]
=== Logic of belief and knowledge ===
* [http://en.wikipedia.org/wiki/Resource_Description_Framework RDF wikipedia]
* [http://www.w3.org/TR/rdf-primer/ w3c RDF primer]
* [http://en.wikipedia.org/wiki/RDFa wiki intro]
* [[Media:portaalidekoosvoime.ppt]] or as pdf [[Media:portaalidekoosvoime.pdf]]
* [http://www.w3.org/2006/07/SWD/RDFa/primer/ w3c rdfa primer]
* [http://tripletalk.wordpress.com/2011/01/25/rdfa-deployment-across-the-web/ rdfa and microformats deployment now]
* [http://www.microformats.org microformats: a similar technology]
* [http://www.w3.org/TR/2003/NOTE-lbase-20031010/ Hayes & Guha Lbase]


* [[Media:Ijcai93.pdf|Ijcai93.pdf]] Overview.


For exam: understand referential transparency and core ideas about encoding belief and knowledge. Modal logic not necessary for the exam.


=== Rules in logic: apps in planning, robotics, ... ===
== Lecture block 4: indexes and search ==


Lecture material:
=== 27. April. Indexes: intro and mainstream ===
* [[Media:Kr_rules1.pdf|Rules and logic: refresher]] and as [[Media:Kr_rules1.ppt|ppt]]


Topics:


* propositional, first order, higher order
Traditional database indexes incl B+ tree:
* semantics/models
* [http://www.lambda.ee/index.php/Pilt:IndexNotes04.ppt G. Molina lecture 4]
* proofs and deriving new information
* automated reasoning


See also:
Hash indexes:
* [http://en.wikipedia.org/wiki/Hash_table wiki].


* [[Media:Geoffreasoningnotes.pdf|Geoff reasoning course notes: just for background, no questions in exam]]
Bitmap indexes:
* [http://www.ee.pdx.edu/~mperkows/May4/005.Ch11-planning.ppt planning and blocks world]
* [http://en.wikipedia.org/wiki/Bitmap_index wiki].
* [[blocks world axiom/query examples from lecture]]


=== Various rule/logic languages ===
For exam: understand the core usage scenarios and be able to create small examples.


Lecture material:
=== 4. May Multi-field and geoindexes, fulltext and term indexes ===
* [[Media:Kr_rdfs.pdf|RDFS: rdf schema]] and as [[Media:Kr_rdfs.ppt|ppt]]
* [[Notes about RDF and OWL: logical meaning]]


Several languages:
* A few words about both multi-field indexes and geoindexes.
* RDFS
* Fulltext indexes: [http://www.lambda.ee/index.php/Pilt:Fulltext_indexes.ppt good intro and overview]
* OWL
* Term indexes: [http://www.lambda.ee/index.php/Pilt:Mccuneindexing.pdf mccune paper]
* KIF
* CL
* ontologies
* wordnet
* cyc
* restricted english systems


See also:
NB! Presentation: geoindexes, 4. May, Daniel.
* [http://en.wikipedia.org/wiki/Resource_Description_Framework RDF wikipedia]
* [http://www.w3.org/TR/rdf-primer/ w3c RDF primer]
* [http://en.wikipedia.org/wiki/Web_Ontology_Language OWL wikipedia]
* [http://www.w3.org/TR/owl-guide/ w3c OWL guide]
* [http://cl.tamu.edu/ CL main page],  
* [http://attempto.ifi.unizh.ch/site/pubs/papers/phd_kaljurand.pdf Attempto restricted english]


=== Scraping semistructured data ===
not part of exam:


* Example for background: http://www.sightsmap.com/
=== 4. May: Fancier term indexes ===


=== Time, context, metainformation ===
* Path indexes


=== Probabilistic and nonmonotonic reasoning ===
* On from here.


See also:
=== 11 May: Nosql indexes plus information about the exam ===
* [http://plato.stanford.edu/entries/logic-nonmonotonic/ nonmonotonic logic]
* [http://en.wikipedia.org/wiki/Default_logic default logic]


=== Personalized search and recommendations ===
* Document bases
* Graph bases


=== Fast search of facts and rules using indexing ===
NB! Indexing in Graph and Document (nosql) databases. 11. May: Vassili.


* [http://www.lambda.ee/index.php/Pilt:IndexNotes04.ppt G. Molina lecture 4]
NB! Last day for the practical work presentations.
* [http://www.lambda.ee/index.php/Pilt:IndexNotes05.ppt G. Molina lecture 5]
* [http://www.lambda.ee/index.php/Pilt:Fulltext_indexes.ppt good intro and overview]
* [http://www.lambda.ee/index.php/Pilt:Mccuneindexing.pdf mccune paper about term indexes]


=== Learning ===
* About the exam.


See also:
== 18. May: guest lecture ==


* [http://www.cs.toronto.edu/~hinton/csc2515/notes/lec1.ppt intro]
Kalle Tomingas. Impact analysis and ontologies.  
* [http://gamma.cs.unc.edu/rob-course/public_html/LEC/MachineLearningMethods.ppt robots/cars]
* [http://www.cs.toronto.edu/~hinton/csc2515/notes/lec4.ppt backpropagation]
* [http://www.cs.toronto.edu/~hinton/csc2515/lectures.html good set of lecture materials]
* [http://www.cs.cmu.edu/~tom/mlbook-chapter-slides.html another good full set of lecture materials]
** sh [http://www.cs.cmu.edu/afs/cs.cmu.edu/project/theo-20/www/mlbook/ch4.pdf neural netw chapter]


http://demo.dlineage.com


<!--
<!--
386. rida: 352. rida:


-->
-->
[[teadmised_note]]


[[Category:Kursus]]
[[Category:Kursus]]
<!--
<pre>
<owl:Class rdf:ID="sugu">
  <rdfs:subClassOf>
    <owl:Restriction>
      <owl:onProperty rdf:resource="sugu" />
      <owl:allValuesFrom rdf:resource="sugu" />
    </owl:Restriction>
  </rdfs:subClassOf>
</owl:Class> 
</pre>
-->

Viimane redaktsioon: 30. jaanuar 2019, kell 09:14

teadmised
teadmised

Code: ITV0060
Link: http://www.lambda.ee/index.php/Teadmiste_otsing,_formaliseerimine_ja_hoidmine or http://www.lambda.ee/index/itv0060
Lecturer: Tanel Tammet
Contact: tanel.tammet@ttu.ee, 6203457, TTÜ ICT-426
Archives of previous years: 2014, 2013, 2012, 2011, 2010, 2009, 2008, 2007, 2006, older.

NB! This is an archive from 2015

Exam

Date and time (two separate occasions):

  • 2. June (Tuesday) at 10:00 in room SOC-214 (Majandus- ja sotsiaalteaduste maja)
  • 9. June (Tuesday) at 10:00 in room SOC-210 (Majandus- ja sotsiaalteaduste maja)

4 questions, one for each block. Each question is a small task in encoding information. No programming excersices.

  • 1. task: encoding data in a conventional relational database in RDF, inventing suitable ID-s, converting selects with where conditions and joins to first order rule form.
  • 2. task: encoding data/rules in RDFs (perhaps additionally in RDFa). Writing 1st order rules about the meaning of RDFs built-in axioms and deriving data using RDFs built-in axioms. Concrete derivation systems are not required: just deriving correct information. It is also important to understand ways to uri-s as global id-s and encoding data in both html metainfo part and in pages, using RDFa.
  • 3. task: encoding uncertain knowledge. You will first have to encode a small scenario using several different systems: default logic, probabilistic logic, fuzzy logic. Second, you have to show what can you derive using one or another formalism.


  • 4. tasks: building indexes. Given a small dataset(s), you will have to build several indexes. Which indexes: trie indexes for words in a text, discrimination tree indexes for generalizations of terms, path indexes for concretisations of terms. For the latter two, use the mccune paper

NB! The information in the following "this is in exam ..." is old and not relevant.

Time, place, result

Semester: spring
Grading: exam

Lectures: every Monday 16:00-17.30, room ICT-A1
Practical work: Mondays on odd weeks (2. Feb, 16 Feb, ...) 17:45-19:15 room ICT-403
Practical work will give 40% and exam 60% of points underlying the final grade. The exam will consist of several small excercises.

Focus

The main focus of the course is on KR (knowledge representation): how to represent nontrivial information in programs and databases, how to build and use indexes for efficient search through large sets of knowledge. Check out this description.

The course contains four blocks built on each other:

  • Background and basics. Representing simple facts.
  • Representing rules.
  • Time, planning and uncertain knowledge.
  • Indexes and search.

Books to use:

Observe that a noticeable part of the course contents are not covered by these books: use the course materials and links to papers, standards and tutorials provided.

Practical work

There are three labs: the first two are obligatory, third is optional. The labs have to be presented to the prof and all students present at labwork.

First lab 2015

The goal of the first lab is to write a software system able to scrape factual raw data about an address (like "Akadeemia tee 12, Tallinn") given to the program.

Basically, search for web pages containing the name or similar names and extract relevant words from the list you create: words closer to the name and with more occurrences are also more important/better match. Titles/headlines above the word are also more likely to be important. It may be a good idea to do some searches together with interesting keywords already. Also, whenever you do a search / pull a page, it is a good idea to store the search result / html source in a file to avoid exchausting your search quota and just wasting bandwidth and time,

It makes sense to search adresses in several ways: googling, binging plus searching from known real estate portals.

Deadline: 16. March (after this there will be a penalty).

See also notes for KR lab 1.

Second lab 2015

The goal of the second lab is to write and use rules to categorize/tag addresses according to the data obtained during the first lab. An address should get a number of tags with numerical indicators showing our trust in that the tag really applies to the address, plus (sometimes) a number indicating the degree to which the tag applies.


See details for KR lab 2.

Third lab 2014

This lab is optional and will simply give as many points as lab 1 or 2 towards the final result and the grade: practical wotrk 60% and exam 40%.

The goal of the lab is to use wordnet or teksaurus to create an additional ruleset and to use that in addition to your own rules in lab 2.

Lab ideas

Student ideas are also welcome: have to agree with Tanel first.

A cool idea is to investigate:



Lecture block 1: basics and representing simple facts.

Lecture 1 (2. Feb) Overview of the course. Background and basics. First lab.

Lecture materials:

Lecture 2 (9 Feb) Programming and databases. SQL: meaning and representation of facts

Lecture materials:

Lecture 3 (16 Feb) HTML annotations. Microformats, microdata, RDFa

Two student presentations ca 30 min each:

  • Microformats, microdata, RDFa: Madis Taimre
  • Facebook open graph: Madis Allikmaa

Lecture materials:

Understand main parts:

Lecture 4 (2. March) RDF last parts + RDFS start

Continuing with RDF:

Starting RDFS:

Choosing the student presentations from:

Lecture block 2: representing rules

Lecture 5 (9. March) Student presentations on NELL and nosql databases + intro to the second lab

Student lectures on the theme of data and knowledge extraction from the web:

  • Nell and systems like Nell (are there any? Check also ConceptNet): Andrei. See some details about Nell as a nice example.
  • Student presentations chosen from Nosql links and notes:

Understand RDFS:

Lecture material:

Additional details (not part of exam):

Lecture 6 (16. March) continuing RDFs and looking into other KR languages

Lecture material:

Additional details (not part of exam):


Important KR languages:

Lecture 7 (23. March): OWL and ontologies

Owl background: description logics (not part of exam )

Understand basics of owl:

not part of exam:

Notes about RDF and OWL: logical meaning

Start looking at interesting ontologies:

Lecture 8 (30 March): Restricted english

Attempto details not necessary for the exam.

Lecture block 3: time, planning and uncertain knowledge

Rules in planning and robotics

Lecture material:

See also (not part of exam):

Logic for uncertain knowledge

For the exam: you should be able to create and solve small examples with default logic.

Fuzzy and probabilistic logic

For the exam: understand the differences between fuzzy and probabilistic logic and be able to present small examples.

Logic of belief and knowledge

For exam: understand referential transparency and core ideas about encoding belief and knowledge. Modal logic not necessary for the exam.

Lecture block 4: indexes and search

27. April. Indexes: intro and mainstream

Traditional database indexes incl B+ tree:

Hash indexes:

Bitmap indexes:

For exam: understand the core usage scenarios and be able to create small examples.

4. May Multi-field and geoindexes, fulltext and term indexes

NB! Presentation: geoindexes, 4. May, Daniel.

not part of exam:

4. May: Fancier term indexes

  • Path indexes
  • On from here.

11 May: Nosql indexes plus information about the exam

  • Document bases
  • Graph bases

NB! Indexing in Graph and Document (nosql) databases. 11. May: Vassili.

NB! Last day for the practical work presentations.

  • About the exam.

18. May: guest lecture

Kalle Tomingas. Impact analysis and ontologies.

http://demo.dlineage.com


teadmised_note

Pärit leheküljelt "http://www.lambda.ee/w/index.php?title=Teadmiste_otsing,_formaliseerimine_ja_hoidmine&oldid=15265"