Teadmiste otsing, formaliseerimine ja hoidmine: erinevus redaktsioonide vahel

Allikas: Lambda
Mine navigeerimisribale Mine otsikasti
← Vanem muudatus
Resümee puudub
 
(ei näidata sama kasutaja 218 vahepealset redaktsiooni)
8. rida: 8. rida:
Link: http://www.lambda.ee/index.php/Teadmiste_otsing,_formaliseerimine_ja_hoidmine  or http://www.lambda.ee/index/itv0060<br>
Link: http://www.lambda.ee/index.php/Teadmiste_otsing,_formaliseerimine_ja_hoidmine  or http://www.lambda.ee/index/itv0060<br>
Lecturer: Tanel Tammet<br>
Lecturer: Tanel Tammet<br>
Contact: tanel.tammet@ttu.ee, 6203457, TTÜ AK223<br>
Contact: tanel.tammet@ttu.ee, 6203457, TTÜ ICT-426<br>
Archives of previous years: [[itv0060 2012 arhiiv|2012]], [[itv0060 2011 arhiiv|2011]], [[itv0060 20010 arhiiv|2010]], [[itv0060 2009 arhiiv|2009]], [[itv0060 2008 arhiiv|2008]], [[itv0060 2007 arhiiv|2007]],  [[itv0060 2006 arhiiv|2006]], [http:/cs.ttu.ee/kursused/wav4130/ older].
Archives of previous years: [[itv0060 2014 arhiiv|2014]], [[itv0060 2013 arhiiv|2013]], [[itv0060 2012 arhiiv|2012]], [[itv0060 2011 arhiiv|2011]], [[itv0060 20010 arhiiv|2010]], [[itv0060 2009 arhiiv|2009]], [[itv0060 2008 arhiiv|2008]], [[itv0060 2007 arhiiv|2007]],  [[itv0060 2006 arhiiv|2006]], [http://cs.ttu.ee/kursused/wav4130/ older].
</td>
</td>
</tr>
</tr>
</table>
</table>


== Muutused loengukohas ==
== <font color="red">NB! This is an archive from 2015</font> ==


Loengud toimuvad nüüd ja edaspidi uues IT majas Akadeemia 15, 4 korrus arvutiteaduse seminariruumis 411.
== Exam ==
 
Date and time (two separate occasions):
 
* 2. June (Tuesday) at 10:00 in room <b>SOC-214</b> (Majandus- ja sotsiaalteaduste maja)
* 9. June (Tuesday) at 10:00 in room <b>SOC-210</b> (Majandus- ja sotsiaalteaduste maja)
 
4 questions, one for each block. Each question is a small task in encoding information. No programming excersices.
 
* 1. task: encoding data in a conventional relational database in RDF, inventing suitable ID-s, converting selects with where conditions and joins to first order rule form.
 
* 2. task: encoding data/rules in RDFs (perhaps additionally in RDFa). Writing 1st order rules about the meaning of RDFs built-in axioms and deriving data using RDFs built-in axioms. Concrete derivation systems are not required: just deriving correct information. It is also important to understand ways to uri-s as global id-s and encoding data in both html metainfo part and in pages, using RDFa.
 
* 3. task: encoding uncertain knowledge. You will first have to encode a small scenario using several different systems: default logic, probabilistic logic, fuzzy logic. Second, you have to show what can you derive using one or another formalism.
 
 
* 4. tasks: building indexes. Given a small dataset(s), you will have to build several indexes. Which indexes: trie indexes for words in a text, discrimination tree indexes for generalizations of terms, path indexes for concretisations of terms. For the latter two, use the [http://www.lambda.ee/index.php/Pilt:Mccuneindexing.pdf mccune paper]
 
NB! The information in the following "this is in exam ..." is old and not relevant.


== Time, place, result ==
== Time, place, result ==
22. rida: 40. rida:
Semester: spring<br>
Semester: spring<br>
Grading: exam<br>
Grading: exam<br>
Points: 3.5<br>


Lectures: every Monday 16:00-17.30, room IT-140.<br>
Lectures: every Monday 16:00-17.30, room ICT-A1<br>
Practical work: Thursdays on odd weeks 16:00-17:30 room IT-213D,  IT-213H.<br>
Practical work: Mondays on odd weeks (2. Feb, 16 Feb, ...) 17:45-19:15 room ICT-403<br>
Practical work will give 40% and exam 60% of points underlying the final grade.
Practical work will give 40% and exam 60% of points underlying the final grade.  
The exam will consist of several small excercises.
 
== Focus ==


The main focus of the course is on KR ([https://en.wikipedia.org/wiki/Knowledge_representation knowledge representation]): how to represent nontrivial information in programs and databases, how to build and use indexes for efficient search through large sets of knowledge. Check out [http://www.lisperati.com/tellstuff/ this description].
The course contains four blocks built on each other:
* Background and basics. Representing simple facts.
* Representing rules.
* Time, planning and uncertain knowledge.
* Indexes and search.
Books to use:
* Ca half of the course themes are covered in [http://www.sciencedirect.com/science/book/9781558609327 this book] with freely accessible pdf-s.
* The freely accesible pdf of the [http://ii.fmph.uniba.sk/kri/KRhandbook.pdf handbook if knowledge representation] gives a detailed and thorough coverage of the subject; far more than necessary for the course.
* Interesting to browse: [http://www.aaai.org/Library/KR/kr12contents.php recent conference proceedings]
* Interesting to browse: [http://www.cse.buffalo.edu/~shapiro/Courses/CSE563/Slides/krrSlides.pdf course materials], [http://web.stanford.edu/class/cs227/ course materials]
Observe that a noticeable part of the course contents are not covered by these books: use the course materials and links to papers, standards and tutorials provided.


== Practical work ==
== Practical work ==


There are three labs. All labs have to be presented to the prof and all students present at labwork.
There are three labs: the first two are obligatory, third is optional. The labs have to be presented to the prof and all students present at labwork.
 
=== First lab 2013 ===


The main goal is to select a domain and a task for the second lab. What to search, from where, etc. You should experiment
=== First lab 2015 ===
with scraping, search and apis to find out what you can really get, in practice. The ideas, experiments and the final
plan should be presented. Might be useful to have a look at:
* http://www.programmableweb.com/
* http://www.webdesignerdepot.com/2011/07/40-useful-apis-for-web-designers-and-developers/
* http://stackoverflow.com/questions/6084096/what-free-web-search-apis-are-available
* http://commoncrawl.org/data/accessing-the-data/
* https://www.facebook.com/SocialNetworkAnalysis
* http://www.sightsmap.com
* http://wordnetweb.princeton.edu
* http://dbpedia.org/About
* https://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=Paris%20Hilton


The goal of the first lab is to write a software system able to scrape factual raw data about an address (like "Akadeemia tee 12, Tallinn") given to the program.


Deadline: 21. February.
Basically, search for web pages containing the name or similar names and extract relevant words from the list you create: words closer to the name and with more occurrences are also more important/better match. Titles/headlines above the word are also more likely to be important. It may be a good idea to do some searches together with interesting keywords already. Also, whenever you do a search / pull a page, it is a good idea to store the search result / html source in a file to avoid exchausting your search quota and just wasting bandwidth and time,


=== Second lab 2013 ===
It makes sense to search adresses in several ways: googling, binging plus searching from known real estate portals.


The main goal is to develop an app which scrapes/searches potentially uncertain data from the web and creates a representation
Deadline: <b>16. March</b> (after this there will be a penalty).
of that data in a machine-processable form (json, csv, xml etc). You can scrape html pages, google, bing, wolfram alpha results
and/or use any apis like twitter, google places, google plus etc. It is best to have a combination of these.


Deadline: end of March.
See also [[notes for KR lab 1]].


=== Third lab 2013 ===
=== Second lab 2015 ===


The main goal of the third lab is to develop a ruleset for generating new interesting facts from the data obtained from the second
The goal of the second lab is to write and use rules to categorize/tag addresses according to the data obtained during the first lab. An address should get a number of tags with numerical indicators showing our trust in that the tag really applies to the address, plus (sometimes) a number indicating the degree to which the tag applies.
lab. The ruleset has to be encoded in a format suitable for some first-order based logical prover and then used
with the prover to actually derive new facts.


Deadline: end of the spring semester.


[[ITV006 third lab 2013]]
See [[details for KR lab 2]].


== Old versions: 2012 ==
=== Third lab 2014 ===


The old versions are here for inspiration and initial ideas.
This lab is <b>optional</b> and will simply give as many points as lab 1 or 2 towards the final result and the grade: practical wotrk 60% and exam 40%.


=== Old version, not for 2013: First lab ===
The goal of the lab is to use [http://wordnet.princeton.edu/ wordnet] or [http://www.cl.ut.ee/ressursid/teksaurus/ teksaurus] to create an additional ruleset and to use that
in addition to your own rules in lab 2.


The main goal of the first lab is to web-scrape data from the public sources using search engines and simple text analysis/statistics. 
=== Lab ideas ===


Details in [[Teadmised first lab 2012|Teadmised first lab 2012]]. The lab should be completed by March, later presentation will cut the points to half.
* [[KR lab idea by Priit Järv]]


=== Old version, not for 2013: Second lab ===
Student ideas are also welcome: have to agree with Tanel first.


The main goal of the second lab is to augment data obtained from the first lab by writing and using probabilistic rules with a rule engine.
A cool idea is to investigate:
* http://threatpost.com/fbstalker-automates-facebook-graph-search-data-mining
* https://github.com/milo2012/osintstalker


Details about second lab will be available on beginning of April. Previous year lab can be seen here: [[Teadmised second lab 2012]]. You may also have a look at the 2011 version of the lab [[Teadmised second lab 2011]].


=== Old version, not for 2013: Optional third lab ===


The optional third lab can be done in parallel with the second. It will be a relatively complex lab: successful passing will give additional 20 points, thus the weight of the exam is diminished from 60% to 40% of the overall score.


What to do: use Attempto to create a system such that you can write rules and/or facts and/or queries for the second lab in restricted english. These restricted english sentences have to be automatically converted to the form usable by Otter. It is a good idea to avoid probabilities in these rules, since it would be very hard to encode probabilities using attempto.
== Lecture block 1: basics and representing simple facts. ==
See: http://attempto.ifi.uzh.ch/site/ , http://attempto.ifi.uzh.ch/ape/


== Lectures passed  ==


=== 1. Declarative and procedural representations ===
=== Lecture 1 (2. Feb) Overview of the course. Background and basics. First lab.  ===


Lecture materials:
Lecture materials:
* [[Media:Kr intro.ppt | Intro lecture: declarative and procedural representations]]
* [[Media:Kr intro.ppt | Intro lecture: declarative and procedural representations]]
* Core logic refresher.
* Details of the first lab.
=== Lecture 2 (9 Feb) Programming and databases. SQL: meaning and representation of facts ===
* Encoding data in programming languages.
* [[Media:Kr_plain_data_db_logic.ppt | relation of plain data in databases to logic & representing complex structures in databases ]]
* [[Media:Kr_plain_data_db_logic.ppt | relation of plain data in databases to logic & representing complex structures in databases ]]
 
* Core ideas of non-relational databases, mostly RDF
=== 2. Databases: meaning and representation of facts ===


Lecture materials:
Lecture materials:
* [[Media:Kr_schemaless.pdf|key/value pairs and rdf, compared to the relational model]]


* Start with [[Media:kr_rdf_schemalessdb_3_2.ppt|key/value pairs and rdf, compared to the relational model]]
* [[Nosql links and notes]]


=== 3. NoSQL databases ===
=== Lecture 3 (16 Feb) HTML annotations. Microformats, microdata, RDFa  ===


* [[Nosql links and notes]]
Two student presentations ca 30 min each:


 
* Microformats, microdata, RDFa: Madis Taimre
=== 4. HTML annotations. RDFa, reading and using RDF data from web pages. Starting RDFs ===
* Facebook open graph: Madis Allikmaa


Lecture materials:
Lecture materials:


* RDFa and RDFs: [[Media:Kr_annot_rdfa_rdfs.ppt]] or as  [[Media:kr_annot_rdfa_rdfs.pdf]] 
* [[Media:Kr_html_annotations.ppt|Html annotations]]


Understand main parts:
Understand main parts:


* [https://support.google.com/webmasters/answer/99170?hl=en Google rich snippets]
* [http://en.wikipedia.org/wiki/RDFa wiki intro]
* [http://en.wikipedia.org/wiki/RDFa wiki intro]
* [[Media:portaalidekoosvoime.ppt]] or as pdf [[Media:portaalidekoosvoime.pdf]]
* [http://www.w3.org/TR/rdfa-lite/ RDFa lite] ja [http://www.w3.org/TR/xhtml-rdfa-primer/ w3c rdfa primer]
* [http://www.w3.org/TR/xhtml-rdfa-primer/ w3c rdfa primer
* [http://ogp.me/ open graph protocol (i.e. Facebook stuff)]
* [http://microformats.org/ microformats]
* [[Media:portaalidekoosvoime.ppt]] or as pdf [[Media:portaalidekoosvoime.pdf]]


=== Lecture 4 (2. March) RDF last parts + RDFS start ===


=== 5. RDFs continued. Starting OWL ===
Continuing with RDF:


* [[Notes about RDF and OWL: logical meaning]]
* Continue with the lecture material [[Media:kr_rdf_schemalessdb_3_2.ppt|key/value pairs and rdf, compared to the relational model]]
* Check out http://www.w3.org/RDF/ and http://www.w3.org/standards/techs/rdf#w3c_all
* And read more from http://www.w3.org/TR/rdf11-primer/ and http://www.w3.org/TR/rdf11-new/


Understand RDFS:
Starting RDFS:


* [http://en.wikipedia.org/wiki/RDF_schema RDFs wikipedia]
* Start with the lecture material [[Media:Kr_rdfs.pdf|RDFS: rdf schema]] and as [[Media:Kr_rdfs.ppt|ppt]]
* [http://www.w3.org/TR/rdf-primer/ w3c RDF primer] see chapter "5. Defining RDF Vocabularies: RDF Schema"
* Read [http://en.wikipedia.org/wiki/RDF_schema RDFs wikipedia]
* Check out [http://www.w3.org/TR/rdf-schema/ w3c RDF schema]


Understand basics of owl:
Choosing the student presentations from:  
* [[Nosql links and notes]]


* [http://en.wikipedia.org/wiki/Web_Ontology_Language OWL wikipedia]
== Lecture block 2: representing rules ==
* [http://www.w3.org/TR/owl2-overview/ w3c OWL version 2 guide]
* [http://www.w3.org/TR/owl2-primer/ w3c OWL version 2 primer]
* [http://www.w3.org/2007/OWL/wiki/Profiles#Reasoning_in_OWL_2_RL_and_RDF_Graphs_using_Rules Owl2 wiki rule semantics] 


Owl background: description logics


* [http://www.inf.unibz.it/~franconi/dl/course/ detailed course in description logics]
=== Lecture 5 (9. March) Student presentations on NELL and nosql databases + intro to the second lab ===


Start looking at interesting ontologies:
Student lectures on the theme of data and knowledge extraction from the web:
* Nell and systems like Nell (are there any? Check also ConceptNet): Andrei. See some details about [http://rtw.ml.cmu.edu/rtw/ Nell] as a nice example.
* Student presentations chosen from [[Nosql links and notes]]:
** Rait: docs.mongodb.org/manual/  (tehtud)
** Raigo: http://www.neo4j.org/ (tehtud)


* [http://www.ontologyportal.org/ SUMO]
Understand RDFS:


Lecture material:


* Continue with [[Media:Kr_rdfs.pdf|RDFS: rdf schema]] and as [[Media:Kr_rdfs.ppt|ppt]]


<!--
Additional details (not part of exam):


=== RDF, RDFS, OWL and conversion to 1st order logic ===
* [http://en.wikipedia.org/wiki/RDF_schema RDFs wikipedia]
* [http://www.w3.org/TR/rdf-schema/ w3c RDF schema]


Lecture material:
=== Lecture 6 (16. March) continuing RDFs and looking into other KR languages ===
* [[Media:Kr_rdfs.pdf|RDFS: rdf schema]] and as [[Media:Kr_rdfs.ppt|ppt]]


Understand RDF and RDFS:


* [http://en.wikipedia.org/wiki/Resource_Description_Framework RDF wikipedia]
Lecture material:
* [http://www.w3.org/TR/rdf-primer/ w3c RDF primer]


Understand basics of owl:
* Continue with [[Media:Kr_rdfs.pdf|RDFS: rdf schema]] and as [[Media:Kr_rdfs.ppt|ppt]]


* [http://en.wikipedia.org/wiki/Web_Ontology_Language OWL wikipedia]
Additional details (not part of exam):
* [http://www.w3.org/TR/owl-guide/ w3c OWL guide]


Notes from lecture about RDF relation to logic:
* [http://en.wikipedia.org/wiki/RDF_schema RDFs wikipedia]
* [[Notes about RDF and OWL: logical meaning]]
* [http://www.w3.org/TR/rdf-schema/ w3c RDF schema]


Understand a bit of:
* [http://www.w3.org/TR/2003/NOTE-lbase-20031010/ Hayes & Guha Lbase]


-->
Important KR languages:


== Rough plan for future lectures ==
* RuleML [http://wiki.ruleml.org/index.php/RuleML_Home RuleML ]
* OWL [http://en.wikipedia.org/wiki/Web_Ontology_Language OWL wikipedia],  [http://www.w3.org/TR/owl-guide/ w3c OWL guide]
* KIF
* CL [http://iso-commonlogic.org CL main page],
* ontologies
* wordnet
* cyc
* TPTP language [http://www.cs.miami.edu/~tptp/TPTP/QuickGuide/ TPTP]
<!-- * restricted english systems [http://attempto.ifi.unizh.ch/site/pubs/papers/phd_kaljurand.pdf Attempto restricted english] -->


=== Rules in logic: apps in planning, robotics, ... ===
=== Lecture 7 (23. March): OWL and ontologies ===


Lecture material:
Owl background: description logics (not part of exam )
* [[Media:Kr_rules1.pdf|Rules and logic: refresher]] and as [[Media:Kr_rules1.ppt|ppt]]


See also:
* [https://www.inf.unibz.it/~franconi/dl/course/slides/prop-DL/propositional-dl.pdf description logic intro]
* [http://disi.unitn.it/~ldkr/ml2013/slides/12.DL.ppt brief alternative intro to description logic]
* [http://www.inf.unibz.it/~franconi/dl/course/ detailed course in description logics] (not necessary for exam)


* [[Media:Geoffreasoningnotes.pdf|Geoff reasoning course notes: just for background, no questions in exam]]
Understand basics of owl:
* [http://www.ee.pdx.edu/~mperkows/May4/005.Ch11-planning.ppt planning and blocks world]
* [[blocks world axiom/query examples from lecture]]


===  First order logic in query engines and planning ===
* [http://en.wikipedia.org/wiki/Web_Ontology_Language OWL wikipedia]


* [[Media:Geoffreasoningnotes.pdf|Geoff reasoning course notes: just for background, no questions in exam]]
not part of exam:
* [http://www.ee.pdx.edu/~mperkows/May4/005.Ch11-planning.ppt planning and blocks world]
* [[blocks world axiom/query examples from lecture]]


* [http://www.w3.org/TR/owl2-overview/ w3c OWL version 2 guide]
* [http://www.w3.org/TR/owl2-primer/ w3c OWL version 2 primer]
* [http://www.w3.org/2007/OWL/wiki/Profiles#Reasoning_in_OWL_2_RL_and_RDF_Graphs_using_Rules Owl2 wiki rule semantics] 


=== Data extraction ===
[[Notes about RDF and OWL: logical meaning]]


Web mining by Nell:
Start looking at interesting ontologies:
* [http://rtw.ml.cmu.edu/rtw/ Nell]


===  Restricted english jms ===
* [http://wordnet.princeton.edu/ Wordnet]
* [http://www.cl.ut.ee/ressursid/teksaurus/ Teksaurus]
* [http://conceptnet5.media.mit.edu/ Conceptnet]
* [https://schema.org/docs/full.html schema.org]
* [http://www.ontologyportal.org/ SUMO]


* [http://attempto.ifi.unizh.ch/site/pubs/papers/phd_kaljurand.pdf Attempto restricted english]
===  Lecture 8 (30 March): Restricted english ===


=== Uncertain knowledge logic ===
* [http://attempto.ifi.uzh.ch/site/pubs/ Attempto restricted english] as intro use [http://attempto.ifi.uzh.ch/site/talks/files/Talk.Stanford.05.pdf this talk]
* [http://inform7.com/ Inform] system for interactive fiction, vt ka [http://inform-fiction.org/manual/html/contents.html vanemat raamatut]


* [http://plato.stanford.edu/entries/logic-nonmonotonic/ nonmonotonic logic]
Attempto details not necessary for the exam.
* [http://en.wikipedia.org/wiki/Default_logic default logic]
* [http://sc.snu.ac.kr/courses/2008/spring/pg/krr/Lectures/Lecture11/Lec11.ppt Reasoning with Conflicting Knowledge]


=== Indexes ===
== Lecture block 3: time, planning and uncertain knowledge ==


Traditional database indexes incl B+ tree:
* [http://www.lambda.ee/index.php/Pilt:IndexNotes04.ppt G. Molina lecture 4]


Bitmap indexes:
=== Rules in planning and robotics ===
* [http://en.wikipedia.org/wiki/Bitmap_index wiki]


Hash indexes:
Lecture material:
* [http://en.wikipedia.org/wiki/Hash_table wiki]
* [[Media:Kr_rules1.pdf|Rules and logic: refresher]] and as [[Media:Kr_rules1.ppt|ppt]]
* [[blocks world axiom/query examples from lecture]]


Fulltext indexes:
See also (not part of exam):
* [http://www.lambda.ee/index.php/Pilt:Fulltext_indexes.ppt good intro and overview]


Term indexes:
* [[Media:Geoffreasoningnotes.pdf|Geoff reasoning course notes: just for background, no questions in exam]]
Understand basics of discrimination tree trie-type term indexes and path indexes from
* [http://www.ee.pdx.edu/~mperkows/May4/005.Ch11-planning.ppt planning and blocks world]
* [http://www.lambda.ee/index.php/Pilt:Mccuneindexing.pdf mccune paper]


== Course structure ==
=== Logic for uncertain knowledge ===


We start with representing and converting facts, continue to rules, first monotonic, then probabilistic and nonmonotonic, then look into search and indexing of complex information, and finally, some learning.
* [http://plato.stanford.edu/entries/logic-nonmonotonic/ nonmonotonic logic] Not necessary for exam.
* [http://en.wikipedia.org/wiki/Default_logic default logic] For exam: main material for default logic.
<!--
* [http://sc.snu.ac.kr/courses/2008/spring/pg/krr/Lectures/Lecture11/Lec11.ppt Reasoning with Conflicting Knowledge] Helpful additional material.
-->


=== Intro and paradigms of knowledge representation ===
For the exam: you should be able to create and solve small examples with default logic.


Lecture materials:
=== Fuzzy and probabilistic logic ===
* [[Media:Kr intro.ppt | Intro lecture: declarative and procedural representations]]
* [http://dl.dropbox.com/u/11794296/roboswarm_saso_2.pdf roboswarm example of combining dec and proc]


For KR, declarative and procedural rep see:
* [[Media:Uncertain_prob_fuzzy.ppt‎|Uncertain_prob_fuzzy.ppt‎]] Intro.
* [http://en.wikipedia.org/wiki/Knowledge_representation Short wiki overview of knowledge representation]
* [[Media:Vienna_tanel.pdf|Vienna_tanel_2.pdf]] Additional examples and combining.
* [http://groups.csail.mit.edu/medg/ftp/psz/k-rep.html Long and philosophical MIT overview of knowledge representation]
* [http://www.cse.buffalo.edu/~rapaport/676/F01/proc.decl.html Pointers to classical papers]
* [http://www.cogarch.org/index.php/Main_Page A Survey of Cognitive and Agent Architectures]
* [ftp://reports.stanford.edu/pub/cstr/reports/cs/tr/76/581/CS-TR-76-581.pdf Classical KRL paper from 1977: Bobrow & Winograd]
Web mining by Nell:
* [http://rtw.ml.cmu.edu/rtw/ Nell]
* [http://www.cs.cmu.edu/~rcwang/ Richard Wang]
* [http://www.boowa.com/ boowa extractor]


=== Databases: meaning and representation of facts ===
For the exam: understand the differences between fuzzy and probabilistic logic and be able to present small examples.


Lecture material:
=== Logic of belief and knowledge ===


<!-- * [[Media:Kr_plain_data_db_logic_draft.ppt|logic and databases intro draft]] -->
* [[Media:Ijcai93.pdf|Ijcai93.pdf]] Overview.
* [[Media:Kr_plain_data_db_logic.ppt | relation of plain data in databases to logic & representing complex structures in databases ]]
<!-- * [[Media:Kr_prop_val_pairs.ppt|property/value pairs]] -->
* [[Media:Kr_rdf_schemalessdb_3.pdf|key/value pairs and rdf, compared to the relational model]]


See also about databases and logic:
For exam: understand referential transparency and core ideas about encoding belief and knowledge. Modal logic not necessary for the exam.


* [http://www.cs.rice.edu/~tlogic/Database/all-lectures.pdf Logic and Database Queries (readable)]
== Lecture block 4: indexes and search ==
* [http://magenta.ruc.dk/dat_en/research/reports/127.pdf logic in databases workshop (complex stuff)]


See also about RDF and related:
=== 27. April. Indexes: intro and mainstream ===


* [http://cs.ttu.ee/kursused/wav4130/rdftutapp.pdf rdf tutorial addendum for logic and dbases]
* [http://en.wikipedia.org/wiki/Resource_Description_Framework RDF wikipedia]
* [http://www.w3.org/TR/rdf-primer/ w3c RDF primer]
* [http://en.wikipedia.org/wiki/RDFa wiki intro]
* [[Media:portaalidekoosvoime.ppt]] or as pdf [[Media:portaalidekoosvoime.pdf]]
* [http://www.w3.org/2006/07/SWD/RDFa/primer/ w3c rdfa primer]
* [http://tripletalk.wordpress.com/2011/01/25/rdfa-deployment-across-the-web/ rdfa and microformats deployment now]
* [http://www.microformats.org microformats: a similar technology]
* [http://www.w3.org/TR/2003/NOTE-lbase-20031010/ Hayes & Guha Lbase]


=== Rules in logic: apps in planning, robotics, ... ===
Traditional database indexes incl B+ tree:
* [http://www.lambda.ee/index.php/Pilt:IndexNotes04.ppt G. Molina lecture 4]


Lecture material:
Hash indexes:
* [[Media:Kr_rules1.pdf|Rules and logic: refresher]] and as [[Media:Kr_rules1.ppt|ppt]]
* [http://en.wikipedia.org/wiki/Hash_table wiki].


Topics:
Bitmap indexes:
* [http://en.wikipedia.org/wiki/Bitmap_index wiki].


* propositional, first order, higher order
For exam: understand the core usage scenarios and be able to create small examples.
* semantics/models
* proofs and deriving new information
* automated reasoning


See also:
=== 4. May Multi-field and geoindexes, fulltext and term indexes ===


* [[Media:Geoffreasoningnotes.pdf|Geoff reasoning course notes: just for background, no questions in exam]]
* A few words about both multi-field indexes and geoindexes.
* [http://www.ee.pdx.edu/~mperkows/May4/005.Ch11-planning.ppt planning and blocks world]
* Fulltext indexes: [http://www.lambda.ee/index.php/Pilt:Fulltext_indexes.ppt good intro and overview]
* [[blocks world axiom/query examples from lecture]]
* Term indexes: [http://www.lambda.ee/index.php/Pilt:Mccuneindexing.pdf mccune paper]
 
=== Various rule/logic languages ===
 
Lecture material:
* [[Media:Kr_rdfs.pdf|RDFS: rdf schema]] and as [[Media:Kr_rdfs.ppt|ppt]]
* [[Notes about RDF and OWL: logical meaning]]
 
Several languages:
* RDFS
* OWL
* KIF
* CL
* ontologies
* wordnet
* cyc
* restricted english systems


See also:
NB! Presentation: geoindexes, 4. May, Daniel.
* [http://en.wikipedia.org/wiki/Resource_Description_Framework RDF wikipedia]
* [http://www.w3.org/TR/rdf-primer/ w3c RDF primer]
* [http://en.wikipedia.org/wiki/Web_Ontology_Language OWL wikipedia]
* [http://www.w3.org/TR/owl-guide/ w3c OWL guide]
* [http://cl.tamu.edu/ CL main page],  
* [http://attempto.ifi.unizh.ch/site/pubs/papers/phd_kaljurand.pdf Attempto restricted english]


=== Scraping semistructured data ===
not part of exam:


* Example for background: http://www.sightsmap.com/
=== 4. May: Fancier term indexes ===


=== Time, context, metainformation ===
* Path indexes


=== Probabilistic and nonmonotonic reasoning ===
* On from here.


See also:
=== 11 May: Nosql indexes plus information about the exam ===
* [http://plato.stanford.edu/entries/logic-nonmonotonic/ nonmonotonic logic]
* [http://en.wikipedia.org/wiki/Default_logic default logic]


=== Personalized search and recommendations ===
* Document bases
* Graph bases


=== Fast search of facts and rules using indexing ===
NB! Indexing in Graph and Document (nosql) databases. 11. May: Vassili.


* [http://www.lambda.ee/index.php/Pilt:IndexNotes04.ppt G. Molina lecture 4]
NB! Last day for the practical work presentations.
* [http://www.lambda.ee/index.php/Pilt:IndexNotes05.ppt G. Molina lecture 5]
* [http://www.lambda.ee/index.php/Pilt:Fulltext_indexes.ppt good intro and overview]
* [http://www.lambda.ee/index.php/Pilt:Mccuneindexing.pdf mccune paper about term indexes]


=== Learning ===
* About the exam.


See also:
== 18. May: guest lecture ==


* [http://www.cs.toronto.edu/~hinton/csc2515/notes/lec1.ppt intro]
Kalle Tomingas. Impact analysis and ontologies.  
* [http://gamma.cs.unc.edu/rob-course/public_html/LEC/MachineLearningMethods.ppt robots/cars]
* [http://www.cs.toronto.edu/~hinton/csc2515/notes/lec4.ppt backpropagation]
* [http://www.cs.toronto.edu/~hinton/csc2515/lectures.html good set of lecture materials]
* [http://www.cs.cmu.edu/~tom/mlbook-chapter-slides.html another good full set of lecture materials]
** sh [http://www.cs.cmu.edu/afs/cs.cmu.edu/project/theo-20/www/mlbook/ch4.pdf neural netw chapter]


http://demo.dlineage.com


<!--
<!--
367. rida: 352. rida:


-->
-->
[[teadmised_note]]


[[Category:Kursus]]
[[Category:Kursus]]
<!--
<pre>
<owl:Class rdf:ID="sugu">
  <rdfs:subClassOf>
    <owl:Restriction>
      <owl:onProperty rdf:resource="sugu" />
      <owl:allValuesFrom rdf:resource="sugu" />
    </owl:Restriction>
  </rdfs:subClassOf>
</owl:Class> 
</pre>
-->

Viimane redaktsioon: 30. jaanuar 2019, kell 09:14

teadmised
teadmised

Code: ITV0060
Link: http://www.lambda.ee/index.php/Teadmiste_otsing,_formaliseerimine_ja_hoidmine or http://www.lambda.ee/index/itv0060
Lecturer: Tanel Tammet
Contact: tanel.tammet@ttu.ee, 6203457, TTÜ ICT-426
Archives of previous years: 2014, 2013, 2012, 2011, 2010, 2009, 2008, 2007, 2006, older.

NB! This is an archive from 2015

Exam

Date and time (two separate occasions):

  • 2. June (Tuesday) at 10:00 in room SOC-214 (Majandus- ja sotsiaalteaduste maja)
  • 9. June (Tuesday) at 10:00 in room SOC-210 (Majandus- ja sotsiaalteaduste maja)

4 questions, one for each block. Each question is a small task in encoding information. No programming excersices.

  • 1. task: encoding data in a conventional relational database in RDF, inventing suitable ID-s, converting selects with where conditions and joins to first order rule form.
  • 2. task: encoding data/rules in RDFs (perhaps additionally in RDFa). Writing 1st order rules about the meaning of RDFs built-in axioms and deriving data using RDFs built-in axioms. Concrete derivation systems are not required: just deriving correct information. It is also important to understand ways to uri-s as global id-s and encoding data in both html metainfo part and in pages, using RDFa.
  • 3. task: encoding uncertain knowledge. You will first have to encode a small scenario using several different systems: default logic, probabilistic logic, fuzzy logic. Second, you have to show what can you derive using one or another formalism.


  • 4. tasks: building indexes. Given a small dataset(s), you will have to build several indexes. Which indexes: trie indexes for words in a text, discrimination tree indexes for generalizations of terms, path indexes for concretisations of terms. For the latter two, use the mccune paper

NB! The information in the following "this is in exam ..." is old and not relevant.

Time, place, result

Semester: spring
Grading: exam

Lectures: every Monday 16:00-17.30, room ICT-A1
Practical work: Mondays on odd weeks (2. Feb, 16 Feb, ...) 17:45-19:15 room ICT-403
Practical work will give 40% and exam 60% of points underlying the final grade. The exam will consist of several small excercises.

Focus

The main focus of the course is on KR (knowledge representation): how to represent nontrivial information in programs and databases, how to build and use indexes for efficient search through large sets of knowledge. Check out this description.

The course contains four blocks built on each other:

  • Background and basics. Representing simple facts.
  • Representing rules.
  • Time, planning and uncertain knowledge.
  • Indexes and search.

Books to use:

Observe that a noticeable part of the course contents are not covered by these books: use the course materials and links to papers, standards and tutorials provided.

Practical work

There are three labs: the first two are obligatory, third is optional. The labs have to be presented to the prof and all students present at labwork.

First lab 2015

The goal of the first lab is to write a software system able to scrape factual raw data about an address (like "Akadeemia tee 12, Tallinn") given to the program.

Basically, search for web pages containing the name or similar names and extract relevant words from the list you create: words closer to the name and with more occurrences are also more important/better match. Titles/headlines above the word are also more likely to be important. It may be a good idea to do some searches together with interesting keywords already. Also, whenever you do a search / pull a page, it is a good idea to store the search result / html source in a file to avoid exchausting your search quota and just wasting bandwidth and time,

It makes sense to search adresses in several ways: googling, binging plus searching from known real estate portals.

Deadline: 16. March (after this there will be a penalty).

See also notes for KR lab 1.

Second lab 2015

The goal of the second lab is to write and use rules to categorize/tag addresses according to the data obtained during the first lab. An address should get a number of tags with numerical indicators showing our trust in that the tag really applies to the address, plus (sometimes) a number indicating the degree to which the tag applies.


See details for KR lab 2.

Third lab 2014

This lab is optional and will simply give as many points as lab 1 or 2 towards the final result and the grade: practical wotrk 60% and exam 40%.

The goal of the lab is to use wordnet or teksaurus to create an additional ruleset and to use that in addition to your own rules in lab 2.

Lab ideas

Student ideas are also welcome: have to agree with Tanel first.

A cool idea is to investigate:



Lecture block 1: basics and representing simple facts.

Lecture 1 (2. Feb) Overview of the course. Background and basics. First lab.

Lecture materials:

Lecture 2 (9 Feb) Programming and databases. SQL: meaning and representation of facts

Lecture materials:

Lecture 3 (16 Feb) HTML annotations. Microformats, microdata, RDFa

Two student presentations ca 30 min each:

  • Microformats, microdata, RDFa: Madis Taimre
  • Facebook open graph: Madis Allikmaa

Lecture materials:

Understand main parts:

Lecture 4 (2. March) RDF last parts + RDFS start

Continuing with RDF:

Starting RDFS:

Choosing the student presentations from:

Lecture block 2: representing rules

Lecture 5 (9. March) Student presentations on NELL and nosql databases + intro to the second lab

Student lectures on the theme of data and knowledge extraction from the web:

  • Nell and systems like Nell (are there any? Check also ConceptNet): Andrei. See some details about Nell as a nice example.
  • Student presentations chosen from Nosql links and notes:

Understand RDFS:

Lecture material:

Additional details (not part of exam):

Lecture 6 (16. March) continuing RDFs and looking into other KR languages

Lecture material:

Additional details (not part of exam):


Important KR languages:

Lecture 7 (23. March): OWL and ontologies

Owl background: description logics (not part of exam )

Understand basics of owl:

not part of exam:

Notes about RDF and OWL: logical meaning

Start looking at interesting ontologies:

Lecture 8 (30 March): Restricted english

Attempto details not necessary for the exam.

Lecture block 3: time, planning and uncertain knowledge

Rules in planning and robotics

Lecture material:

See also (not part of exam):

Logic for uncertain knowledge

For the exam: you should be able to create and solve small examples with default logic.

Fuzzy and probabilistic logic

For the exam: understand the differences between fuzzy and probabilistic logic and be able to present small examples.

Logic of belief and knowledge

For exam: understand referential transparency and core ideas about encoding belief and knowledge. Modal logic not necessary for the exam.

Lecture block 4: indexes and search

27. April. Indexes: intro and mainstream

Traditional database indexes incl B+ tree:

Hash indexes:

Bitmap indexes:

For exam: understand the core usage scenarios and be able to create small examples.

4. May Multi-field and geoindexes, fulltext and term indexes

NB! Presentation: geoindexes, 4. May, Daniel.

not part of exam:

4. May: Fancier term indexes

  • Path indexes
  • On from here.

11 May: Nosql indexes plus information about the exam

  • Document bases
  • Graph bases

NB! Indexing in Graph and Document (nosql) databases. 11. May: Vassili.

NB! Last day for the practical work presentations.

  • About the exam.

18. May: guest lecture

Kalle Tomingas. Impact analysis and ontologies.

http://demo.dlineage.com


teadmised_note

Pärit leheküljelt "http://www.lambda.ee/w/index.php?title=Teadmiste_otsing,_formaliseerimine_ja_hoidmine&oldid=15265"