Teadmiste otsing, formaliseerimine ja hoidmine: erinevus redaktsioonide vahel

Allikas: Lambda
Mine navigeerimisribale Mine otsikasti
← Vanem muudatus
Resümee puudub
 
(ei näidata 2 kasutaja 367 vahepealset redaktsiooni)
1. rida: 1. rida:
<table>
<tr>
<td>
[[Pilt:Mephistophiles small.gif| left|teadmised]]
</td>
<td>
Code: <b>ITV0060</b><br>
Code: <b>ITV0060</b><br>
Link: http://www.lambda.ee/index.php/Teadmiste_otsing,_formaliseerimine_ja_hoidmine  or http://www.lambda.ee/index/itv0060<br>
Link: http://www.lambda.ee/index.php/Teadmiste_otsing,_formaliseerimine_ja_hoidmine  or http://www.lambda.ee/index/itv0060<br>
Alternative names and codes:
* Tehisintellekti algoritmid ITT0020
* Intelligent Systems ITX8075
Lecturer: Tanel Tammet<br>
Lecturer: Tanel Tammet<br>
Contact: tammet@staff.ttu.ee, 6203457, TTÜ AK223<br>
Contact: tanel.tammet@ttu.ee, 6203457, TTÜ ICT-426<br>
Archives of previous years: [[itv0060 2008 arhiiv|2008]], [[itv0060 2007 arhiiv|2007]],  [[itv0060 2006 arhiiv|2006]], [http:/cs.ttu.ee/kursused/wav4130/ older].
Archives of previous years: [[itv0060 2014 arhiiv|2014]], [[itv0060 2013 arhiiv|2013]], [[itv0060 2012 arhiiv|2012]], [[itv0060 2011 arhiiv|2011]], [[itv0060 20010 arhiiv|2010]], [[itv0060 2009 arhiiv|2009]], [[itv0060 2008 arhiiv|2008]], [[itv0060 2007 arhiiv|2007]],  [[itv0060 2006 arhiiv|2006]], [http://cs.ttu.ee/kursused/wav4130/ older].
</td>
</tr>
</table>


== <font color="red">Eksamiajad</font> ==
== <font color="red">NB! This is an archive from 2015</font> ==


Kaks alternatiivset aega:
== Exam ==


* 28.05.2009 kell 9:00 ruumis I-202
Date and time (two separate occasions):
* 05.06.2009 kell 9:00 ruumis VI-121


NB! vahepeal siin lehel seisnud alternatiiv kuupäeval 21.05 oli näpuviga, sel kuupäeval eksamit ei ole.
* 2. June (Tuesday) at 10:00 in room <b>SOC-214</b> (Majandus- ja sotsiaalteaduste maja)
* 9. June (Tuesday) at 10:00 in room <b>SOC-210</b> (Majandus- ja sotsiaalteaduste maja)


== Materials for exam ==
4 questions, one for each block. Each question is a small task in encoding information. No programming excersices.


Use and read the following
* 1. task: encoding data in a conventional relational database in RDF, inventing suitable ID-s, converting selects with where conditions and joins to first order rule form.


=== RDF, RDFS, OWL and conversion to 1st order logic ===
* 2. task: encoding data/rules in RDFs (perhaps additionally in RDFa). Writing 1st order rules about the meaning of RDFs built-in axioms and deriving data using RDFs built-in axioms. Concrete derivation systems are not required: just deriving correct information. It is also important to understand ways to uri-s as global id-s and encoding data in both html metainfo part and in pages, using RDFa.


Understand RDF and RDFS:
* 3. task: encoding uncertain knowledge. You will first have to encode a small scenario using several different systems: default logic, probabilistic logic, fuzzy logic. Second, you have to show what can you derive using one or another formalism.


* [http://en.wikipedia.org/wiki/Resource_Description_Framework RDF wikipedia]
* [http://www.w3.org/TR/rdf-primer/ w3c RDF primer]


Understand basics of owl:
* 4. tasks: building indexes. Given a small dataset(s), you will have to build several indexes. Which indexes: trie indexes for words in a text, discrimination tree indexes for generalizations of terms, path indexes for concretisations of terms. For the latter two, use the [http://www.lambda.ee/index.php/Pilt:Mccuneindexing.pdf mccune paper]
 
NB! The information in the following "this is in exam ..." is old and not relevant.
 
== Time, place, result ==
 
Semester: spring<br>
Grading: exam<br>
 
Lectures: every Monday 16:00-17.30, room ICT-A1<br>
Practical work: Mondays on odd weeks (2. Feb, 16 Feb, ...) 17:45-19:15 room ICT-403<br>
Practical work will give 40% and exam 60% of points underlying the final grade.
The exam will consist of several small excercises.
 
== Focus ==


* [http://en.wikipedia.org/wiki/Web_Ontology_Language OWL wikipedia]
The main focus of the course is on KR ([https://en.wikipedia.org/wiki/Knowledge_representation knowledge representation]): how to represent nontrivial information in programs and databases, how to build and use indexes for efficient search through large sets of knowledge. Check out [http://www.lisperati.com/tellstuff/ this description].
* [http://www.w3.org/TR/owl-guide/ w3c OWL guide]


=== RDFa, reading and using RDF data from web pages ===
The course contains four blocks built on each other:


Understand main parts:
* Background and basics. Representing simple facts.
* Representing rules.
* Time, planning and uncertain knowledge.
* Indexes and search.


* [http://en.wikipedia.org/wiki/RDFa wiki intro]
Books to use:
* [[Media:portaalidekoosvoime.ppt]] or as pdf [[Media:portaalidekoosvoime.pdf]]
* [http://www.w3.org/2006/07/SWD/RDFa/primer/ w3c rdfa primer]
* [http://www.microformats.org microformats: a similar technology]


=== RDF relation to logic ===
* Ca half of the course themes are covered in [http://www.sciencedirect.com/science/book/9781558609327 this book] with freely accessible pdf-s.
* The freely accesible pdf of the [http://ii.fmph.uniba.sk/kri/KRhandbook.pdf handbook if knowledge representation] gives a detailed and thorough coverage of the subject; far more than necessary for the course.
* Interesting to browse: [http://www.aaai.org/Library/KR/kr12contents.php recent conference proceedings]
* Interesting to browse: [http://www.cse.buffalo.edu/~shapiro/Courses/CSE563/Slides/krrSlides.pdf course materials], [http://web.stanford.edu/class/cs227/ course materials]


Understand a bit of:
Observe that a noticeable part of the course contents are not covered by these books: use the course materials and links to papers, standards and tutorials provided.


* [http://www.w3.org/TR/2003/NOTE-lbase-20031010/ Hayes & Guha Lbase]
== Practical work ==


=== Semantics of ordinary relational base ===
There are three labs: the first two are obligatory, third is optional. The labs have to be presented to the prof and all students present at labwork.


* [http://cs.ttu.ee/kursused/wav4130/rdftutapp.pdf rdf tutorial addendum for logic and dbases]
=== First lab 2015 ===


===  First order logic in query engines ===
The goal of the first lab is to write a software system able to scrape factual raw data about an address (like "Akadeemia tee 12, Tallinn") given to the program.


* [http://www.cs.miami.edu/~geoff/Courses/CS63S-05M/CS63SNotes.pdf Geoff intro]
Basically, search for web pages containing the name or similar names and extract relevant words from the list you create: words closer to the name and with more occurrences are also more important/better match. Titles/headlines above the word are also more likely to be important. It may be a good idea to do some searches together with interesting keywords already. Also, whenever you do a search / pull a page, it is a good idea to store the search result / html source in a file to avoid exchausting your search quota and just wasting bandwidth and time,


===  Different languages for 1st order logic: KIF, CL, restricted english jms ===
It makes sense to search adresses in several ways: googling, binging plus searching from known real estate portals.


* [http://cl.tamu.edu/ CL main page],
Deadline: <b>16. March</b> (after this there will be a penalty).
* [http://attempto.ifi.unizh.ch/site/pubs/papers/phd_kaljurand.pdf Attempto restricted english]


See also [[notes for KR lab 1]].


=== Representing knowledge and belief ===
=== Second lab 2015 ===


The goal of the second lab is to write and use rules to categorize/tag addresses according to the data obtained during the first lab. An address should get a number of tags with numerical indicators showing our trust in that the tag really applies to the address, plus (sometimes) a number indicating the degree to which the tag applies.


===  Fulltext indexes ===


See [[details for KR lab 2]].


=== Term indexes ===
=== Third lab 2014 ===


== Learning (only for ITX8075) ==
This lab is <b>optional</b> and will simply give as many points as lab 1 or 2 towards the final result and the grade: practical wotrk 60% and exam 40%.


== Time, place, result ==
The goal of the lab is to use [http://wordnet.princeton.edu/ wordnet] or [http://www.cl.ut.ee/ressursid/teksaurus/ teksaurus] to create an additional ruleset and to use that
in addition to your own rules in lab 2.


Semester: spring<br>
=== Lab ideas ===
Grading: exam<br>
Points: 3.5<br>


Lectures: Every friday 8.00, room IT-140<br>
* [[KR lab idea by Priit Järv]]
Practical work: every second (even weeks) Friday 10.00 room  IT-213D<br>


Practical work will give 40% and exam 60% of points underlying the final grade. NB! Course ITV0060 and ITT0020 attendees may optionally do the third lab and get 60% from labs and 40% from exam.
Student ideas are also welcome: have to agree with Tanel first.


== Practical work ==
A cool idea is to investigate:
* http://threatpost.com/fbstalker-automates-facebook-graph-search-data-mining
* https://github.com/milo2012/osintstalker


* For ITV0060 and ITT0020: two labs on the same theme.
* For ITX8075: same two labs plus a third lab below.
* As noted above, course ITV0060 and ITT0020 attendees may optionally do the third lab and get 60% from labs and 40% from exam.




=== First lab ===


Date: 13 March.  
== Lecture block 1: basics and representing simple facts. ==


Implement a system which can read html annotated according to the RDFa standard.
The data read should be saved and printed in several formats:
* csv
* rdf xml syntax


You will have to either find suitable annotated html pages or annotate some existing pages yourself (at least three pages).  
=== Lecture 1 (2. Feb) Overview of the course. Background and basics. First lab. ===


Additional reading:
Lecture materials:
* [[Media:Kr intro.ppt | Intro lecture: declarative and procedural representations]]
* Core logic refresher.
* Details of the first lab.


* http://www.w3.org/TR/rdf-primer/ RDF tutorial first parts
=== Lecture 2 (9 Feb) Programming and databases. SQL: meaning and representation of facts ===
* http://en.wikipedia.org/wiki/RDFa wikipedia article on RDFa
* http://www.youtube.com/watch?v=ldl0m-5zLz4 and http://www.youtube.com/watch?v=mxE3FeOyS-E video tutorials of rdfa
* http://www.w3.org/TR/xhtml-rdfa-primer/
* http://www.w3.org/2006/07/SWD/RDFa/syntax/
* [[Media:portaalidekoosvoime.ppt]] või pdf kujul [[Media:portaalidekoosvoime.pdf]]  (Estonian)


You may also look at some practical non-obligatory programming suggestions in Estonian: [[rdfa esimese praksi soovitusi]].
* Encoding data in programming languages.
* [[Media:Kr_plain_data_db_logic.ppt | relation of plain data in databases to logic & representing complex structures in databases ]]
* Core ideas of non-relational databases, mostly RDF


=== Second lab ===
Lecture materials:


Using the results of first lab for rule-based derivation of new information.
* Start with [[Media:kr_rdf_schemalessdb_3_2.ppt|key/value pairs and rdf, compared to the relational model]]
* [[Nosql links and notes]]


[[Details of the second lab: rule system]]
=== Lecture 3 (16 Feb) HTML annotations. Microformats, microdata, RDFa  ===


=== Third lab ===
Two student presentations ca 30 min each:


This lab is obligatory only for the ITX8075 students obtaining 5 AP from the course! You can do it alone or together with another student: pairs are OK, three or more persons not. Both students must be present while presenting the result.
* Microformats, microdata, RDFa: Madis Taimre
* Facebook open graph: Madis Allikmaa


The lab depends on your background. Concretely, choose the first case which applies to you:
Lecture materials:


* You are studying in the IT department and have never programmed a turn-based two-player thinking game. In this case you should program the five-in-a-row game as explained in [[Praktikumitöö: mõtlemismäng 2005]]. This option is also allowed - but not obligatory - for non-IT students who have never programmed a two-player thinking game before.
* [[Media:Kr_html_annotations.ppt|Html annotations]]
* You are studying in the IT department and have programmed a turn-based two-player thinking game. In this case you should experiment with the [http://attempto.ifi.uzh.ch/site/ Attempto system] in the following way:
**  Create a set of English sentences which covers the meaning of your rules from the second lab as closey as possible.
** Use the Attempto system to transform these sentences into a formal rule system.
** Write a program which would transform the sentences from the Attempto-given rule system into the form acceptable by Otter or Prover9.
* All the other cases: you will have to do a project from the similar [http://ocw.mit.edu/OcwWeb/Electrical-Engineering-and-Computer-Science/6-034Spring-2005/CourseHome/index.htm MIT course]. Download the MIT course materials and choose one of the following projects:
** http://ocw.mit.edu/NR/rdonlyres/Electrical-Engineering-and-Computer-Science/6-034Spring-2005/8C027A7F-EDDC-4E01-BEEF-55C94287ED15/0/dp1.pdf
** http://ocw.mit.edu/NR/rdonlyres/Electrical-Engineering-and-Computer-Science/6-034Spring-2005/27A74D7C-5492-4FC2-977C-00FF670C014A/0/dp2.pdf


== Course structure ==
Understand main parts:


=== Intro and paradigms of knowledge representation ===
* [https://support.google.com/webmasters/answer/99170?hl=en Google rich snippets]
* [http://en.wikipedia.org/wiki/RDFa wiki intro]
* [http://www.w3.org/TR/rdfa-lite/ RDFa lite] ja [http://www.w3.org/TR/xhtml-rdfa-primer/ w3c rdfa primer]
* [http://ogp.me/ open graph protocol (i.e. Facebook stuff)]
* [http://microformats.org/ microformats]
* [[Media:portaalidekoosvoime.ppt]] or as pdf [[Media:portaalidekoosvoime.pdf]]


1 lecture
=== Lecture 4 (2. March) RDF last parts + RDFS start ===


Overview of knowledge representation: http://en.wikipedia.org/wiki/Knowledge_representation
Continuing with RDF:


* Procedural
* Continue with the lecture material [[Media:kr_rdf_schemalessdb_3_2.ppt|key/value pairs and rdf, compared to the relational model]]
* Declarative
* Check out http://www.w3.org/RDF/ and http://www.w3.org/standards/techs/rdf#w3c_all
** logic
* And read more from http://www.w3.org/TR/rdf11-primer/ and http://www.w3.org/TR/rdf11-new/
** relational, network, object databases
** semantic networks
** frame systems
** property/value pairs
** rule/expert systems
** rdfs and owl


=== Core search, search in game trees ===
Starting RDFS:


1 lecture
* Start with the lecture material [[Media:Kr_rdfs.pdf|RDFS: rdf schema]] and as [[Media:Kr_rdfs.ppt|ppt]]
* Read [http://en.wikipedia.org/wiki/RDF_schema RDFs wikipedia]
* Check out [http://www.w3.org/TR/rdf-schema/ w3c RDF schema]


Lecture notes: http://www.lambda.ee/index.php/Pilt:Search.ppt
Choosing the student presentations from:
* [[Nosql links and notes]]


Additional material:
== Lecture block 2: representing rules ==


:* [http://en.wikipedia.org/wiki/Search_algorithm wikipedia article on search algorithms]
:* [http://en.wikipedia.org/wiki/A*_search_algorithm wikipedia article on the A* algorithm]
:* [http://aima.cs.berkeley.edu/newchap05.pdf Norvig AIMA constraint satisfaction chapter]
:* Minimax & alpha-beta: [http://en.wikipedia.org/wiki/Minimax minimax wikipedias], [http://en.wikipedia.org/wiki/Alpha-beta_pruning a-b in wikipedia],  [http://www.ocf.berkeley.edu/~yosenl/extras/alphabeta/alphabeta.html applet demo], [http://www.emunix.emich.edu/~evett/AI/AlphaBeta_movie/sld001.htm a-b ok explanation with]
:* Chinook: http://www.cs.ualberta.ca/~chinook/
:* Ph.D. thesis about game programming: http://fragrieu.free.fr/SearchingForSolutions.pdf


=== Logic ===
=== Lecture 5 (9. March) Student presentations on NELL and nosql databases + intro to the second lab ===


2 lectures
Student lectures on the theme of data and knowledge extraction from the web:
* Nell and systems like Nell (are there any? Check also ConceptNet): Andrei. See some details about [http://rtw.ml.cmu.edu/rtw/ Nell] as a nice example.
* Student presentations chosen from [[Nosql links and notes]]:
** Rait: docs.mongodb.org/manual/  (tehtud)
** Raigo: http://www.neo4j.org/ (tehtud)


* propositional, first order, higher order
Understand RDFS:
* semantics/models
* proofs and deriving new information
* automated reasoning
* relational data and rdf in first order logic


=== Planning  ===
Lecture material:


1 lecture
* Continue with [[Media:Kr_rdfs.pdf|RDFS: rdf schema]] and as [[Media:Kr_rdfs.ppt|ppt]]


logic- and reasoning-based methods
Additional details (not part of exam):


* [http://en.wikipedia.org/wiki/RDF_schema RDFs wikipedia]
* [http://www.w3.org/TR/rdf-schema/ w3c RDF schema]


=== Different property/value pair representations  ===
=== Lecture 6 (16. March) continuing RDFs and looking into other KR languages ===


Methods and their relations


1 lecture
Lecture material:


* association lists
* Continue with [[Media:Kr_rdfs.pdf|RDFS: rdf schema]] and as [[Media:Kr_rdfs.ppt|ppt]]
* google base
* RDF
* XML


=== Specific versions of 1st order logic and rule systems ===
Additional details (not part of exam):


1 lecture
* [http://en.wikipedia.org/wiki/RDF_schema RDFs wikipedia]
* [http://www.w3.org/TR/rdf-schema/ w3c RDF schema]


Important KR languages:
* RuleML [http://wiki.ruleml.org/index.php/RuleML_Home RuleML ]
* OWL [http://en.wikipedia.org/wiki/Web_Ontology_Language OWL wikipedia],  [http://www.w3.org/TR/owl-guide/ w3c OWL guide]
* KIF  
* KIF  
* CL  
* CL [http://iso-commonlogic.org CL main page],
* ontologies
* ontologies
* wordnet
* wordnet
* cyc
* cyc
* restricted english systems
* TPTP language [http://www.cs.miami.edu/~tptp/TPTP/QuickGuide/ TPTP]
<!-- * restricted english systems [http://attempto.ifi.unizh.ch/site/pubs/papers/phd_kaljurand.pdf Attempto restricted english] -->
 
=== Lecture 7 (23. March): OWL and ontologies ===
 
Owl background: description logics (not part of exam )
 
* [https://www.inf.unibz.it/~franconi/dl/course/slides/prop-DL/propositional-dl.pdf description logic intro]
* [http://disi.unitn.it/~ldkr/ml2013/slides/12.DL.ppt brief alternative intro to description logic]
* [http://www.inf.unibz.it/~franconi/dl/course/ detailed course in description logics] (not necessary for exam)
 
Understand basics of owl:
 
* [http://en.wikipedia.org/wiki/Web_Ontology_Language OWL wikipedia]
 
not part of exam:
 
* [http://www.w3.org/TR/owl2-overview/ w3c OWL version 2 guide]
* [http://www.w3.org/TR/owl2-primer/ w3c OWL version 2 primer]
* [http://www.w3.org/2007/OWL/wiki/Profiles#Reasoning_in_OWL_2_RL_and_RDF_Graphs_using_Rules Owl2 wiki rule semantics] 
 
[[Notes about RDF and OWL: logical meaning]]
 
Start looking at interesting ontologies:
 
* [http://wordnet.princeton.edu/ Wordnet]
* [http://www.cl.ut.ee/ressursid/teksaurus/ Teksaurus]
* [http://conceptnet5.media.mit.edu/ Conceptnet]
* [https://schema.org/docs/full.html schema.org]
* [http://www.ontologyportal.org/ SUMO]
 
===  Lecture 8 (30 March): Restricted english ===
 
* [http://attempto.ifi.uzh.ch/site/pubs/ Attempto restricted english] as intro use [http://attempto.ifi.uzh.ch/site/talks/files/Talk.Stanford.05.pdf this talk]
* [http://inform7.com/ Inform] system for interactive fiction, vt ka [http://inform-fiction.org/manual/html/contents.html vanemat raamatut]
 
Attempto details not necessary for the exam.
 
== Lecture block 3: time, planning and uncertain knowledge ==
 
 
=== Rules in planning and robotics ===
 
Lecture material:
* [[Media:Kr_rules1.pdf|Rules and logic: refresher]] and as [[Media:Kr_rules1.ppt|ppt]]
* [[blocks world axiom/query examples from lecture]]
 
See also (not part of exam):
 
* [[Media:Geoffreasoningnotes.pdf|Geoff reasoning course notes: just for background, no questions in exam]]
* [http://www.ee.pdx.edu/~mperkows/May4/005.Ch11-planning.ppt planning and blocks world]
 
=== Logic for uncertain knowledge ===


* [http://plato.stanford.edu/entries/logic-nonmonotonic/ nonmonotonic logic] Not necessary for exam.
* [http://en.wikipedia.org/wiki/Default_logic default logic] For exam: main material for default logic.
<!--
* [http://sc.snu.ac.kr/courses/2008/spring/pg/krr/Lectures/Lecture11/Lec11.ppt Reasoning with Conflicting Knowledge] Helpful additional material.
-->


=== Details of RDF, RDFS, RDFa ===
For the exam: you should be able to create and solve small examples with default logic.


1 lecture
=== Fuzzy and probabilistic logic ===


=== OWL ===
* [[Media:Uncertain_prob_fuzzy.ppt‎|Uncertain_prob_fuzzy.ppt‎]] Intro.
* [[Media:Vienna_tanel.pdf|Vienna_tanel_2.pdf]] Additional examples and combining.


1 lecture
For the exam: understand the differences between fuzzy and probabilistic logic and be able to present small examples.


=== Representing time, probability, epistemics, context ===
=== Logic of belief and knowledge ===


1 lecture
* [[Media:Ijcai93.pdf|Ijcai93.pdf]] Overview.


=== Search and indexing ===
For exam: understand referential transparency and core ideas about encoding belief and knowledge. Modal logic not necessary for the exam.


2 lecture
== Lecture block 4: indexes and search ==


=== 27. April. Indexes: intro and mainstream ===
Traditional database indexes incl B+ tree:
* [http://www.lambda.ee/index.php/Pilt:IndexNotes04.ppt G. Molina lecture 4]
Hash indexes:
* [http://en.wikipedia.org/wiki/Hash_table wiki].
Bitmap indexes:
* [http://en.wikipedia.org/wiki/Bitmap_index wiki].
For exam: understand the core usage scenarios and be able to create small examples.
=== 4. May Multi-field and geoindexes, fulltext and term indexes ===
* A few words about both multi-field indexes and geoindexes.
* Fulltext indexes: [http://www.lambda.ee/index.php/Pilt:Fulltext_indexes.ppt good intro and overview]
* Term indexes: [http://www.lambda.ee/index.php/Pilt:Mccuneindexing.pdf mccune paper]
NB! Presentation: geoindexes, 4. May, Daniel.
not part of exam:
=== 4. May: Fancier term indexes ===
* Path indexes
* On from here.
=== 11 May: Nosql indexes plus information about the exam ===
* Document bases
* Graph bases
NB! Indexing in Graph and Document (nosql) databases. 11. May: Vassili.
NB! Last day for the practical work presentations.
* About the exam.
== 18. May: guest lecture ==
Kalle Tomingas. Impact analysis and ontologies.
http://demo.dlineage.com
<!--
=== Learning ===
=== Learning ===


1 lecture
-->
 
<!--


Useful materials:
Useful materials:


* http://www.cs.cmu.edu/~tom/mlbook-chapter-slides.html
* [http://www.cs.toronto.edu/~hinton/csc2515/notes/lec1.ppt intro]
* [http://gamma.cs.unc.edu/rob-course/public_html/LEC/MachineLearningMethods.ppt robots/cars]
* [http://www.cs.toronto.edu/~hinton/csc2515/notes/lec4.ppt backpropagation]
* [http://www.cs.toronto.edu/~hinton/csc2515/lectures.html good set of lecture materials]
* [http://www.cs.cmu.edu/~tom/mlbook-chapter-slides.html another good full set of lecture materials]
** sh [http://www.cs.cmu.edu/afs/cs.cmu.edu/project/theo-20/www/mlbook/ch4.pdf neural netw chapter]


-->
[[teadmised_note]]


[[Category:Kursus]]
[[Category:Kursus]]
<!--
<pre>
<owl:Class rdf:ID="sugu">
  <rdfs:subClassOf>
    <owl:Restriction>
      <owl:onProperty rdf:resource="sugu" />
      <owl:allValuesFrom rdf:resource="sugu" />
    </owl:Restriction>
  </rdfs:subClassOf>
</owl:Class> 
</pre>
-->

Viimane redaktsioon: 30. jaanuar 2019, kell 09:14

teadmised
teadmised

Code: ITV0060
Link: http://www.lambda.ee/index.php/Teadmiste_otsing,_formaliseerimine_ja_hoidmine or http://www.lambda.ee/index/itv0060
Lecturer: Tanel Tammet
Contact: tanel.tammet@ttu.ee, 6203457, TTÜ ICT-426
Archives of previous years: 2014, 2013, 2012, 2011, 2010, 2009, 2008, 2007, 2006, older.

NB! This is an archive from 2015

Exam

Date and time (two separate occasions):

  • 2. June (Tuesday) at 10:00 in room SOC-214 (Majandus- ja sotsiaalteaduste maja)
  • 9. June (Tuesday) at 10:00 in room SOC-210 (Majandus- ja sotsiaalteaduste maja)

4 questions, one for each block. Each question is a small task in encoding information. No programming excersices.

  • 1. task: encoding data in a conventional relational database in RDF, inventing suitable ID-s, converting selects with where conditions and joins to first order rule form.
  • 2. task: encoding data/rules in RDFs (perhaps additionally in RDFa). Writing 1st order rules about the meaning of RDFs built-in axioms and deriving data using RDFs built-in axioms. Concrete derivation systems are not required: just deriving correct information. It is also important to understand ways to uri-s as global id-s and encoding data in both html metainfo part and in pages, using RDFa.
  • 3. task: encoding uncertain knowledge. You will first have to encode a small scenario using several different systems: default logic, probabilistic logic, fuzzy logic. Second, you have to show what can you derive using one or another formalism.


  • 4. tasks: building indexes. Given a small dataset(s), you will have to build several indexes. Which indexes: trie indexes for words in a text, discrimination tree indexes for generalizations of terms, path indexes for concretisations of terms. For the latter two, use the mccune paper

NB! The information in the following "this is in exam ..." is old and not relevant.

Time, place, result

Semester: spring
Grading: exam

Lectures: every Monday 16:00-17.30, room ICT-A1
Practical work: Mondays on odd weeks (2. Feb, 16 Feb, ...) 17:45-19:15 room ICT-403
Practical work will give 40% and exam 60% of points underlying the final grade. The exam will consist of several small excercises.

Focus

The main focus of the course is on KR (knowledge representation): how to represent nontrivial information in programs and databases, how to build and use indexes for efficient search through large sets of knowledge. Check out this description.

The course contains four blocks built on each other:

  • Background and basics. Representing simple facts.
  • Representing rules.
  • Time, planning and uncertain knowledge.
  • Indexes and search.

Books to use:

Observe that a noticeable part of the course contents are not covered by these books: use the course materials and links to papers, standards and tutorials provided.

Practical work

There are three labs: the first two are obligatory, third is optional. The labs have to be presented to the prof and all students present at labwork.

First lab 2015

The goal of the first lab is to write a software system able to scrape factual raw data about an address (like "Akadeemia tee 12, Tallinn") given to the program.

Basically, search for web pages containing the name or similar names and extract relevant words from the list you create: words closer to the name and with more occurrences are also more important/better match. Titles/headlines above the word are also more likely to be important. It may be a good idea to do some searches together with interesting keywords already. Also, whenever you do a search / pull a page, it is a good idea to store the search result / html source in a file to avoid exchausting your search quota and just wasting bandwidth and time,

It makes sense to search adresses in several ways: googling, binging plus searching from known real estate portals.

Deadline: 16. March (after this there will be a penalty).

See also notes for KR lab 1.

Second lab 2015

The goal of the second lab is to write and use rules to categorize/tag addresses according to the data obtained during the first lab. An address should get a number of tags with numerical indicators showing our trust in that the tag really applies to the address, plus (sometimes) a number indicating the degree to which the tag applies.


See details for KR lab 2.

Third lab 2014

This lab is optional and will simply give as many points as lab 1 or 2 towards the final result and the grade: practical wotrk 60% and exam 40%.

The goal of the lab is to use wordnet or teksaurus to create an additional ruleset and to use that in addition to your own rules in lab 2.

Lab ideas

Student ideas are also welcome: have to agree with Tanel first.

A cool idea is to investigate:



Lecture block 1: basics and representing simple facts.

Lecture 1 (2. Feb) Overview of the course. Background and basics. First lab.

Lecture materials:

Lecture 2 (9 Feb) Programming and databases. SQL: meaning and representation of facts

Lecture materials:

Lecture 3 (16 Feb) HTML annotations. Microformats, microdata, RDFa

Two student presentations ca 30 min each:

  • Microformats, microdata, RDFa: Madis Taimre
  • Facebook open graph: Madis Allikmaa

Lecture materials:

Understand main parts:

Lecture 4 (2. March) RDF last parts + RDFS start

Continuing with RDF:

Starting RDFS:

Choosing the student presentations from:

Lecture block 2: representing rules

Lecture 5 (9. March) Student presentations on NELL and nosql databases + intro to the second lab

Student lectures on the theme of data and knowledge extraction from the web:

  • Nell and systems like Nell (are there any? Check also ConceptNet): Andrei. See some details about Nell as a nice example.
  • Student presentations chosen from Nosql links and notes:

Understand RDFS:

Lecture material:

Additional details (not part of exam):

Lecture 6 (16. March) continuing RDFs and looking into other KR languages

Lecture material:

Additional details (not part of exam):


Important KR languages:

Lecture 7 (23. March): OWL and ontologies

Owl background: description logics (not part of exam )

Understand basics of owl:

not part of exam:

Notes about RDF and OWL: logical meaning

Start looking at interesting ontologies:

Lecture 8 (30 March): Restricted english

Attempto details not necessary for the exam.

Lecture block 3: time, planning and uncertain knowledge

Rules in planning and robotics

Lecture material:

See also (not part of exam):

Logic for uncertain knowledge

For the exam: you should be able to create and solve small examples with default logic.

Fuzzy and probabilistic logic

For the exam: understand the differences between fuzzy and probabilistic logic and be able to present small examples.

Logic of belief and knowledge

For exam: understand referential transparency and core ideas about encoding belief and knowledge. Modal logic not necessary for the exam.

Lecture block 4: indexes and search

27. April. Indexes: intro and mainstream

Traditional database indexes incl B+ tree:

Hash indexes:

Bitmap indexes:

For exam: understand the core usage scenarios and be able to create small examples.

4. May Multi-field and geoindexes, fulltext and term indexes

NB! Presentation: geoindexes, 4. May, Daniel.

not part of exam:

4. May: Fancier term indexes

  • Path indexes
  • On from here.

11 May: Nosql indexes plus information about the exam

  • Document bases
  • Graph bases

NB! Indexing in Graph and Document (nosql) databases. 11. May: Vassili.

NB! Last day for the practical work presentations.

  • About the exam.

18. May: guest lecture

Kalle Tomingas. Impact analysis and ontologies.

http://demo.dlineage.com


teadmised_note

Pärit leheküljelt "http://www.lambda.ee/w/index.php?title=Teadmiste_otsing,_formaliseerimine_ja_hoidmine&oldid=15265"