Teadmiste otsing, formaliseerimine ja hoidmine: erinevus redaktsioonide vahel

Allikas: Lambda
Mine navigeerimisribale Mine otsikasti
← Vanem muudatus
Resümee puudub
 
(ei näidata 2 kasutaja 381 vahepealset redaktsiooni)
1. rida: 1. rida:
<table>
<tr>
<td>
[[Pilt:Mephistophiles small.gif| left|teadmised]]
</td>
<td>
Code: <b>ITV0060</b><br>
Code: <b>ITV0060</b><br>
Link: http://www.lambda.ee/index.php/Teadmiste_otsing,_formaliseerimine_ja_hoidmine  or http://www.lambda.ee/index/itv0060<br>
Link: http://www.lambda.ee/index.php/Teadmiste_otsing,_formaliseerimine_ja_hoidmine  or http://www.lambda.ee/index/itv0060<br>
Alternative names and codes:
* Tehisintellekti algoritmid ITT0020
* Intelligent Systems ITX8075
Lecturer: Tanel Tammet<br>
Lecturer: Tanel Tammet<br>
Contact: tammet@staff.ttu.ee, 6203457, TTÜ AK223<br>
Contact: tanel.tammet@ttu.ee, 6203457, TTÜ ICT-426<br>
Archives of previous years: [[itv0060 2008 arhiiv|2008]], [[itv0060 2007 arhiiv|2007]],  [[itv0060 2006 arhiiv|2006]], [http:/cs.ttu.ee/kursused/wav4130/ older].
Archives of previous years: [[itv0060 2014 arhiiv|2014]], [[itv0060 2013 arhiiv|2013]], [[itv0060 2012 arhiiv|2012]], [[itv0060 2011 arhiiv|2011]], [[itv0060 20010 arhiiv|2010]], [[itv0060 2009 arhiiv|2009]], [[itv0060 2008 arhiiv|2008]], [[itv0060 2007 arhiiv|2007]],  [[itv0060 2006 arhiiv|2006]], [http://cs.ttu.ee/kursused/wav4130/ older].
</td>
</tr>
</table>
 
== <font color="red">NB! This is an archive from 2015</font> ==
 
== Exam ==
 
Date and time (two separate occasions):
 
* 2. June (Tuesday) at 10:00 in room <b>SOC-214</b> (Majandus- ja sotsiaalteaduste maja)
* 9. June (Tuesday) at 10:00 in room <b>SOC-210</b> (Majandus- ja sotsiaalteaduste maja)
 
4 questions, one for each block. Each question is a small task in encoding information. No programming excersices.
 
* 1. task: encoding data in a conventional relational database in RDF, inventing suitable ID-s, converting selects with where conditions and joins to first order rule form.
 
* 2. task: encoding data/rules in RDFs (perhaps additionally in RDFa). Writing 1st order rules about the meaning of RDFs built-in axioms and deriving data using RDFs built-in axioms. Concrete derivation systems are not required: just deriving correct information. It is also important to understand ways to uri-s as global id-s and encoding data in both html metainfo part and in pages, using RDFa.
 
* 3. task: encoding uncertain knowledge. You will first have to encode a small scenario using several different systems: default logic, probabilistic logic, fuzzy logic. Second, you have to show what can you derive using one or another formalism.
 
 
* 4. tasks: building indexes. Given a small dataset(s), you will have to build several indexes. Which indexes: trie indexes for words in a text, discrimination tree indexes for generalizations of terms, path indexes for concretisations of terms. For the latter two, use the [http://www.lambda.ee/index.php/Pilt:Mccuneindexing.pdf mccune paper]
 
NB! The information in the following "this is in exam ..." is old and not relevant.


== Time, place, result ==
== Time, place, result ==
12. rida: 40. rida:
Semester: spring<br>
Semester: spring<br>
Grading: exam<br>
Grading: exam<br>
Points: 3.5<br>


Lectures: Every friday 8.00, room IT-140<br>
Lectures: every Monday 16:00-17.30, room ICT-A1<br>
Practical work: every second (even weeks) Friday 10.00 room IT-213D<br>
Practical work: Mondays on odd weeks (2. Feb, 16 Feb, ...) 17:45-19:15 room ICT-403<br>
Practical work will give 40% and exam 60% of points underlying the final grade.
The exam will consist of several small excercises.


Practical work will give 40% and exam 60% of points underlying the final grade. NB! Course ITV0060 and ITT0020 attendees may optionally do the third lab and get 60% from labs and 40% from exam.
== Focus ==
 
The main focus of the course is on KR ([https://en.wikipedia.org/wiki/Knowledge_representation knowledge representation]): how to represent nontrivial information in programs and databases, how to build and use indexes for efficient search through large sets of knowledge. Check out [http://www.lisperati.com/tellstuff/ this description].
 
The course contains four blocks built on each other:
 
* Background and basics. Representing simple facts.
* Representing rules.
* Time, planning and uncertain knowledge.
* Indexes and search.
 
Books to use:
 
* Ca half of the course themes are covered in [http://www.sciencedirect.com/science/book/9781558609327 this book] with freely accessible pdf-s.
* The freely accesible pdf of the [http://ii.fmph.uniba.sk/kri/KRhandbook.pdf handbook if knowledge representation] gives a detailed and thorough coverage of the subject; far more than necessary for the course.
* Interesting to browse: [http://www.aaai.org/Library/KR/kr12contents.php recent conference proceedings]
* Interesting to browse: [http://www.cse.buffalo.edu/~shapiro/Courses/CSE563/Slides/krrSlides.pdf course materials], [http://web.stanford.edu/class/cs227/ course materials]
 
Observe that a noticeable part of the course contents are not covered by these books: use the course materials and links to papers, standards and tutorials provided.


== Practical work ==
== Practical work ==


* For ITV0060 and ITT0020: two labs on the same theme.
There are three labs: the first two are obligatory, third is optional. The labs have to be presented to the prof and all students present at labwork.
* For ITX8075: same two labs plus a third lab below.
* As noted above, course ITV0060 and ITT0020 attendees may optionally do the third lab and get 60% from labs and 40% from exam.


=== First lab 2015 ===


=== First lab ===
The goal of the first lab is to write a software system able to scrape factual raw data about an address (like "Akadeemia tee 12, Tallinn") given to the program.


Date: 13 March.  
Basically, search for web pages containing the name or similar names and extract relevant words from the list you create: words closer to the name and with more occurrences are also more important/better match. Titles/headlines above the word are also more likely to be important. It may be a good idea to do some searches together with interesting keywords already. Also, whenever you do a search / pull a page, it is a good idea to store the search result / html source in a file to avoid exchausting your search quota and just wasting bandwidth and time,


Implement a system which can read html annotated according to the RDFa standard.
It makes sense to search adresses in several ways: googling, binging plus searching from known real estate portals.
The data read should be saved and printed in several formats:
* csv
* rdf xml syntax


You will have to either find suitable annotated html pages or annotate some existing pages yourself (at least three pages).  
Deadline: <b>16. March</b> (after this there will be a penalty).


Additional reading:
See also [[notes for KR lab 1]].


* http://www.w3.org/TR/rdf-primer/ RDF tutorial first parts
=== Second lab 2015 ===
* http://en.wikipedia.org/wiki/RDFa wikipedia article on RDFa
* http://www.youtube.com/watch?v=ldl0m-5zLz4 and http://www.youtube.com/watch?v=mxE3FeOyS-E video tutorials of rdfa
* http://www.w3.org/TR/xhtml-rdfa-primer/
* http://www.w3.org/2006/07/SWD/RDFa/syntax/
* [[Media:portaalidekoosvoime.ppt]] või pdf kujul [[Media:portaalidekoosvoime.pdf]]  (Estonian)


You may also look at some practical non-obligatory programming suggestions in Estonian: [[rdfa esimese praksi soovitusi]].
The goal of the second lab is to write and use rules to categorize/tag addresses according to the data obtained during the first lab. An address should get a number of tags with numerical indicators showing our trust in that the tag really applies to the address, plus (sometimes) a number indicating the degree to which the tag applies.


=== Second lab ===


Using the results of first lab for rule-based derivation of new information.  
See [[details for KR lab 2]].


[[Details of the second lab: rule system]]
=== Third lab 2014 ===


=== Third lab ===
This lab is <b>optional</b> and will simply give as many points as lab 1 or 2 towards the final result and the grade: practical wotrk 60% and exam 40%.


This lab is obligatory only for the ITX8075 students obtaining 5 AP from the course! You can do it alone or together with another student: pairs are OK, three or more persons not. Both students must be present while presenting the result.
The goal of the lab is to use [http://wordnet.princeton.edu/ wordnet] or [http://www.cl.ut.ee/ressursid/teksaurus/ teksaurus] to create an additional ruleset and to use that
in addition to your own rules in lab 2.


The lab depends on your background. Concretely, choose the first case which applies to you:
=== Lab ideas ===


* You are studying in the IT department and have never programmed a turn-based two-player thinking game. In this case you should program the five-in-a-row game as explained in [[Praktikumitöö: mõtlemismäng 2005]]. This option is also allowed - but not obligatory - for non-IT students who have never programmed a two-player thinking game before.
* [[KR lab idea by Priit Järv]]
* You are studying in the IT department and have programmed a turn-based two-player thinking game. In this case you should experiment with the [http://attempto.ifi.uzh.ch/site/ Attempto system] in the following way:
**  Create a set of English sentences which covers the meaning of your rules from the second lab as closey as possible.
** Use the Attempto system to transform these sentences into a formal rule system.
** Write a program which would transform the sentences from the Attempto-given rule system into the form acceptable by Otter or Prover9.
* All the other cases: you will have to do a project from the similar [http://ocw.mit.edu/OcwWeb/Electrical-Engineering-and-Computer-Science/6-034Spring-2005/CourseHome/index.htm MIT course]. Choose one of the following:
** http://ocw.mit.edu/NR/rdonlyres/Electrical-Engineering-and-Computer-Science/6-034Spring-2005/8C027A7F-EDDC-4E01-BEEF-55C94287ED15/0/dp1.pdf
** http://ocw.mit.edu/NR/rdonlyres/Electrical-Engineering-and-Computer-Science/6-034Spring-2005/27A74D7C-5492-4FC2-977C-00FF670C014A/0/dp2.pdf


== Course structure ==
Student ideas are also welcome: have to agree with Tanel first.


=== Intro and paradigms of knowledge representation ===
A cool idea is to investigate:
* http://threatpost.com/fbstalker-automates-facebook-graph-search-data-mining
* https://github.com/milo2012/osintstalker


1 lecture


Overview of knowledge representation: http://en.wikipedia.org/wiki/Knowledge_representation


* Procedural
* Declarative
** logic
** relational, network, object databases
** semantic networks
** frame systems
** property/value pairs
** rule/expert systems
** rdfs and owl


=== Core search, search in game trees ===
== Lecture block 1: basics and representing simple facts. ==


1 lecture


Lecture notes: http://www.lambda.ee/index.php/Pilt:Search.ppt
=== Lecture 1 (2. Feb) Overview of the course. Background and basics. First lab. ===


Additional material:
Lecture materials:
* [[Media:Kr intro.ppt | Intro lecture: declarative and procedural representations]]
* Core logic refresher.
* Details of the first lab.


:* [http://en.wikipedia.org/wiki/Search_algorithm wikipedia article on search algorithms]
=== Lecture 2 (9 Feb) Programming and databases. SQL: meaning and representation of facts ===
:* [http://en.wikipedia.org/wiki/A*_search_algorithm wikipedia article on the A* algorithm]
:* [http://aima.cs.berkeley.edu/newchap05.pdf Norvig AIMA constraint satisfaction chapter]
:* Minimax & alpha-beta: [http://en.wikipedia.org/wiki/Minimax minimax wikipedias], [http://en.wikipedia.org/wiki/Alpha-beta_pruning a-b in wikipedia],  [http://www.ocf.berkeley.edu/~yosenl/extras/alphabeta/alphabeta.html applet demo], [http://www.emunix.emich.edu/~evett/AI/AlphaBeta_movie/sld001.htm a-b ok explanation with]
:* Chinook: http://www.cs.ualberta.ca/~chinook/
:* Ph.D. thesis about game programming: http://fragrieu.free.fr/SearchingForSolutions.pdf


=== Logic ===
* Encoding data in programming languages.
* [[Media:Kr_plain_data_db_logic.ppt | relation of plain data in databases to logic & representing complex structures in databases ]]
* Core ideas of non-relational databases, mostly RDF


2 lectures
Lecture materials:


* propositional, first order, higher order
* Start with [[Media:kr_rdf_schemalessdb_3_2.ppt|key/value pairs and rdf, compared to the relational model]]
* semantics/models
* [[Nosql links and notes]]
* proofs and deriving new information
* automated reasoning
* relational data and rdf in first order logic


=== Planning ===
=== Lecture 3 (16 Feb) HTML annotations. Microformats, microdata, RDFa ===


1 lecture
Two student presentations ca 30 min each:


logic- and reasoning-based methods
* Microformats, microdata, RDFa: Madis Taimre
* Facebook open graph: Madis Allikmaa


Lecture materials:


=== Different property/value pair representations  ===
* [[Media:Kr_html_annotations.ppt|Html annotations]]


Methods and their relations
Understand main parts:


1 lecture
* [https://support.google.com/webmasters/answer/99170?hl=en Google rich snippets]
* [http://en.wikipedia.org/wiki/RDFa wiki intro]
* [http://www.w3.org/TR/rdfa-lite/ RDFa lite] ja [http://www.w3.org/TR/xhtml-rdfa-primer/ w3c rdfa primer]
* [http://ogp.me/ open graph protocol (i.e. Facebook stuff)]
* [http://microformats.org/ microformats]
* [[Media:portaalidekoosvoime.ppt]] or as pdf [[Media:portaalidekoosvoime.pdf]]


* association lists
=== Lecture 4 (2. March) RDF last parts + RDFS start ===
* google base
* RDF
* XML


=== Specific versions of 1st order logic and rule systems ===
Continuing with RDF:


1 lecture
* Continue with the lecture material [[Media:kr_rdf_schemalessdb_3_2.ppt|key/value pairs and rdf, compared to the relational model]]
* Check out http://www.w3.org/RDF/ and http://www.w3.org/standards/techs/rdf#w3c_all
* And read more from http://www.w3.org/TR/rdf11-primer/ and http://www.w3.org/TR/rdf11-new/


Starting RDFS:
* Start with the lecture material [[Media:Kr_rdfs.pdf|RDFS: rdf schema]] and as [[Media:Kr_rdfs.ppt|ppt]]
* Read [http://en.wikipedia.org/wiki/RDF_schema RDFs wikipedia]
* Check out [http://www.w3.org/TR/rdf-schema/ w3c RDF schema]
Choosing the student presentations from:
* [[Nosql links and notes]]
== Lecture block 2: representing rules ==
=== Lecture 5 (9. March) Student presentations on NELL and nosql databases + intro to the second lab ===
Student lectures on the theme of data and knowledge extraction from the web:
* Nell and systems like Nell (are there any? Check also ConceptNet): Andrei. See some details about [http://rtw.ml.cmu.edu/rtw/ Nell] as a nice example.
* Student presentations chosen from [[Nosql links and notes]]:
** Rait: docs.mongodb.org/manual/  (tehtud)
** Raigo: http://www.neo4j.org/ (tehtud)
Understand RDFS:
Lecture material:
* Continue with [[Media:Kr_rdfs.pdf|RDFS: rdf schema]] and as [[Media:Kr_rdfs.ppt|ppt]]
Additional details (not part of exam):
* [http://en.wikipedia.org/wiki/RDF_schema RDFs wikipedia]
* [http://www.w3.org/TR/rdf-schema/ w3c RDF schema]
=== Lecture 6 (16. March) continuing RDFs and looking into other KR languages ===
Lecture material:
* Continue with [[Media:Kr_rdfs.pdf|RDFS: rdf schema]] and as [[Media:Kr_rdfs.ppt|ppt]]
Additional details (not part of exam):
* [http://en.wikipedia.org/wiki/RDF_schema RDFs wikipedia]
* [http://www.w3.org/TR/rdf-schema/ w3c RDF schema]
Important KR languages:
* RuleML [http://wiki.ruleml.org/index.php/RuleML_Home RuleML ]
* OWL [http://en.wikipedia.org/wiki/Web_Ontology_Language OWL wikipedia],  [http://www.w3.org/TR/owl-guide/ w3c OWL guide]
* KIF  
* KIF  
* CL  
* CL [http://iso-commonlogic.org CL main page],
* ontologies
* ontologies
* wordnet
* wordnet
* cyc
* cyc
* restricted english systems
* TPTP language [http://www.cs.miami.edu/~tptp/TPTP/QuickGuide/ TPTP]
<!-- * restricted english systems [http://attempto.ifi.unizh.ch/site/pubs/papers/phd_kaljurand.pdf Attempto restricted english] -->
 
=== Lecture 7 (23. March): OWL and ontologies ===


Owl background: description logics (not part of exam )


=== Details of RDF, RDFS, RDFa ===
* [https://www.inf.unibz.it/~franconi/dl/course/slides/prop-DL/propositional-dl.pdf description logic intro]
* [http://disi.unitn.it/~ldkr/ml2013/slides/12.DL.ppt brief alternative intro to description logic]
* [http://www.inf.unibz.it/~franconi/dl/course/ detailed course in description logics] (not necessary for exam)


1 lecture
Understand basics of owl:


=== OWL ===
* [http://en.wikipedia.org/wiki/Web_Ontology_Language OWL wikipedia]


1 lecture
not part of exam:


=== Representing time, probability, epistemics, context ===
* [http://www.w3.org/TR/owl2-overview/ w3c OWL version 2 guide]
* [http://www.w3.org/TR/owl2-primer/ w3c OWL version 2 primer]
* [http://www.w3.org/2007/OWL/wiki/Profiles#Reasoning_in_OWL_2_RL_and_RDF_Graphs_using_Rules Owl2 wiki rule semantics] 


1 lecture
[[Notes about RDF and OWL: logical meaning]]


=== Search and indexing ===
Start looking at interesting ontologies:


2 lecture
* [http://wordnet.princeton.edu/ Wordnet]
* [http://www.cl.ut.ee/ressursid/teksaurus/ Teksaurus]
* [http://conceptnet5.media.mit.edu/ Conceptnet]
* [https://schema.org/docs/full.html schema.org]
* [http://www.ontologyportal.org/ SUMO]


===  Lecture 8 (30 March): Restricted english ===
* [http://attempto.ifi.uzh.ch/site/pubs/ Attempto restricted english] as intro use [http://attempto.ifi.uzh.ch/site/talks/files/Talk.Stanford.05.pdf this talk]
* [http://inform7.com/ Inform] system for interactive fiction, vt ka [http://inform-fiction.org/manual/html/contents.html vanemat raamatut]
Attempto details not necessary for the exam.
== Lecture block 3: time, planning and uncertain knowledge ==
=== Rules in planning and robotics ===
Lecture material:
* [[Media:Kr_rules1.pdf|Rules and logic: refresher]] and as [[Media:Kr_rules1.ppt|ppt]]
* [[blocks world axiom/query examples from lecture]]
See also (not part of exam):
* [[Media:Geoffreasoningnotes.pdf|Geoff reasoning course notes: just for background, no questions in exam]]
* [http://www.ee.pdx.edu/~mperkows/May4/005.Ch11-planning.ppt planning and blocks world]
=== Logic for uncertain knowledge ===
* [http://plato.stanford.edu/entries/logic-nonmonotonic/ nonmonotonic logic] Not necessary for exam.
* [http://en.wikipedia.org/wiki/Default_logic default logic] For exam: main material for default logic.
<!--
* [http://sc.snu.ac.kr/courses/2008/spring/pg/krr/Lectures/Lecture11/Lec11.ppt Reasoning with Conflicting Knowledge] Helpful additional material.
-->
For the exam: you should be able to create and solve small examples with default logic.
=== Fuzzy and probabilistic logic ===
* [[Media:Uncertain_prob_fuzzy.ppt‎|Uncertain_prob_fuzzy.ppt‎]] Intro.
* [[Media:Vienna_tanel.pdf|Vienna_tanel_2.pdf]] Additional examples and combining.
For the exam: understand the differences between fuzzy and probabilistic logic and be able to present small examples.
=== Logic of belief and knowledge ===
* [[Media:Ijcai93.pdf|Ijcai93.pdf]] Overview.
For exam: understand referential transparency and core ideas about encoding belief and knowledge. Modal logic not necessary for the exam.
== Lecture block 4: indexes and search ==
=== 27. April. Indexes: intro and mainstream ===
Traditional database indexes incl B+ tree:
* [http://www.lambda.ee/index.php/Pilt:IndexNotes04.ppt G. Molina lecture 4]
Hash indexes:
* [http://en.wikipedia.org/wiki/Hash_table wiki].
Bitmap indexes:
* [http://en.wikipedia.org/wiki/Bitmap_index wiki].
For exam: understand the core usage scenarios and be able to create small examples.
=== 4. May Multi-field and geoindexes, fulltext and term indexes ===
* A few words about both multi-field indexes and geoindexes.
* Fulltext indexes: [http://www.lambda.ee/index.php/Pilt:Fulltext_indexes.ppt good intro and overview]
* Term indexes: [http://www.lambda.ee/index.php/Pilt:Mccuneindexing.pdf mccune paper]
NB! Presentation: geoindexes, 4. May, Daniel.
not part of exam:
=== 4. May: Fancier term indexes ===
* Path indexes
* On from here.
=== 11 May: Nosql indexes plus information about the exam ===
* Document bases
* Graph bases
NB! Indexing in Graph and Document (nosql) databases. 11. May: Vassili.
NB! Last day for the practical work presentations.
* About the exam.
== 18. May: guest lecture ==
Kalle Tomingas. Impact analysis and ontologies.
http://demo.dlineage.com
<!--
=== Learning ===
=== Learning ===


1 lecture
-->
 
<!--


[[Category:Kursus]]
Useful materials:
 
* [http://www.cs.toronto.edu/~hinton/csc2515/notes/lec1.ppt intro]
* [http://gamma.cs.unc.edu/rob-course/public_html/LEC/MachineLearningMethods.ppt robots/cars]
* [http://www.cs.toronto.edu/~hinton/csc2515/notes/lec4.ppt backpropagation]
* [http://www.cs.toronto.edu/~hinton/csc2515/lectures.html good set of lecture materials]
* [http://www.cs.cmu.edu/~tom/mlbook-chapter-slides.html another good full set of lecture materials]
** sh [http://www.cs.cmu.edu/afs/cs.cmu.edu/project/theo-20/www/mlbook/ch4.pdf neural netw chapter]


<!--
<pre>
<owl:Class rdf:ID="sugu">
  <rdfs:subClassOf>
    <owl:Restriction>
      <owl:onProperty rdf:resource="sugu" />
      <owl:allValuesFrom rdf:resource="sugu" />
    </owl:Restriction>
  </rdfs:subClassOf>
</owl:Class> 
</pre>
-->
-->
[[teadmised_note]]
[[Category:Kursus]]

Viimane redaktsioon: 30. jaanuar 2019, kell 09:14

teadmised
teadmised

Code: ITV0060
Link: http://www.lambda.ee/index.php/Teadmiste_otsing,_formaliseerimine_ja_hoidmine or http://www.lambda.ee/index/itv0060
Lecturer: Tanel Tammet
Contact: tanel.tammet@ttu.ee, 6203457, TTÜ ICT-426
Archives of previous years: 2014, 2013, 2012, 2011, 2010, 2009, 2008, 2007, 2006, older.

NB! This is an archive from 2015

Exam

Date and time (two separate occasions):

  • 2. June (Tuesday) at 10:00 in room SOC-214 (Majandus- ja sotsiaalteaduste maja)
  • 9. June (Tuesday) at 10:00 in room SOC-210 (Majandus- ja sotsiaalteaduste maja)

4 questions, one for each block. Each question is a small task in encoding information. No programming excersices.

  • 1. task: encoding data in a conventional relational database in RDF, inventing suitable ID-s, converting selects with where conditions and joins to first order rule form.
  • 2. task: encoding data/rules in RDFs (perhaps additionally in RDFa). Writing 1st order rules about the meaning of RDFs built-in axioms and deriving data using RDFs built-in axioms. Concrete derivation systems are not required: just deriving correct information. It is also important to understand ways to uri-s as global id-s and encoding data in both html metainfo part and in pages, using RDFa.
  • 3. task: encoding uncertain knowledge. You will first have to encode a small scenario using several different systems: default logic, probabilistic logic, fuzzy logic. Second, you have to show what can you derive using one or another formalism.


  • 4. tasks: building indexes. Given a small dataset(s), you will have to build several indexes. Which indexes: trie indexes for words in a text, discrimination tree indexes for generalizations of terms, path indexes for concretisations of terms. For the latter two, use the mccune paper

NB! The information in the following "this is in exam ..." is old and not relevant.

Time, place, result

Semester: spring
Grading: exam

Lectures: every Monday 16:00-17.30, room ICT-A1
Practical work: Mondays on odd weeks (2. Feb, 16 Feb, ...) 17:45-19:15 room ICT-403
Practical work will give 40% and exam 60% of points underlying the final grade. The exam will consist of several small excercises.

Focus

The main focus of the course is on KR (knowledge representation): how to represent nontrivial information in programs and databases, how to build and use indexes for efficient search through large sets of knowledge. Check out this description.

The course contains four blocks built on each other:

  • Background and basics. Representing simple facts.
  • Representing rules.
  • Time, planning and uncertain knowledge.
  • Indexes and search.

Books to use:

Observe that a noticeable part of the course contents are not covered by these books: use the course materials and links to papers, standards and tutorials provided.

Practical work

There are three labs: the first two are obligatory, third is optional. The labs have to be presented to the prof and all students present at labwork.

First lab 2015

The goal of the first lab is to write a software system able to scrape factual raw data about an address (like "Akadeemia tee 12, Tallinn") given to the program.

Basically, search for web pages containing the name or similar names and extract relevant words from the list you create: words closer to the name and with more occurrences are also more important/better match. Titles/headlines above the word are also more likely to be important. It may be a good idea to do some searches together with interesting keywords already. Also, whenever you do a search / pull a page, it is a good idea to store the search result / html source in a file to avoid exchausting your search quota and just wasting bandwidth and time,

It makes sense to search adresses in several ways: googling, binging plus searching from known real estate portals.

Deadline: 16. March (after this there will be a penalty).

See also notes for KR lab 1.

Second lab 2015

The goal of the second lab is to write and use rules to categorize/tag addresses according to the data obtained during the first lab. An address should get a number of tags with numerical indicators showing our trust in that the tag really applies to the address, plus (sometimes) a number indicating the degree to which the tag applies.


See details for KR lab 2.

Third lab 2014

This lab is optional and will simply give as many points as lab 1 or 2 towards the final result and the grade: practical wotrk 60% and exam 40%.

The goal of the lab is to use wordnet or teksaurus to create an additional ruleset and to use that in addition to your own rules in lab 2.

Lab ideas

Student ideas are also welcome: have to agree with Tanel first.

A cool idea is to investigate:



Lecture block 1: basics and representing simple facts.

Lecture 1 (2. Feb) Overview of the course. Background and basics. First lab.

Lecture materials:

Lecture 2 (9 Feb) Programming and databases. SQL: meaning and representation of facts

Lecture materials:

Lecture 3 (16 Feb) HTML annotations. Microformats, microdata, RDFa

Two student presentations ca 30 min each:

  • Microformats, microdata, RDFa: Madis Taimre
  • Facebook open graph: Madis Allikmaa

Lecture materials:

Understand main parts:

Lecture 4 (2. March) RDF last parts + RDFS start

Continuing with RDF:

Starting RDFS:

Choosing the student presentations from:

Lecture block 2: representing rules

Lecture 5 (9. March) Student presentations on NELL and nosql databases + intro to the second lab

Student lectures on the theme of data and knowledge extraction from the web:

  • Nell and systems like Nell (are there any? Check also ConceptNet): Andrei. See some details about Nell as a nice example.
  • Student presentations chosen from Nosql links and notes:

Understand RDFS:

Lecture material:

Additional details (not part of exam):

Lecture 6 (16. March) continuing RDFs and looking into other KR languages

Lecture material:

Additional details (not part of exam):


Important KR languages:

Lecture 7 (23. March): OWL and ontologies

Owl background: description logics (not part of exam )

Understand basics of owl:

not part of exam:

Notes about RDF and OWL: logical meaning

Start looking at interesting ontologies:

Lecture 8 (30 March): Restricted english

Attempto details not necessary for the exam.

Lecture block 3: time, planning and uncertain knowledge

Rules in planning and robotics

Lecture material:

See also (not part of exam):

Logic for uncertain knowledge

For the exam: you should be able to create and solve small examples with default logic.

Fuzzy and probabilistic logic

For the exam: understand the differences between fuzzy and probabilistic logic and be able to present small examples.

Logic of belief and knowledge

For exam: understand referential transparency and core ideas about encoding belief and knowledge. Modal logic not necessary for the exam.

Lecture block 4: indexes and search

27. April. Indexes: intro and mainstream

Traditional database indexes incl B+ tree:

Hash indexes:

Bitmap indexes:

For exam: understand the core usage scenarios and be able to create small examples.

4. May Multi-field and geoindexes, fulltext and term indexes

NB! Presentation: geoindexes, 4. May, Daniel.

not part of exam:

4. May: Fancier term indexes

  • Path indexes
  • On from here.

11 May: Nosql indexes plus information about the exam

  • Document bases
  • Graph bases

NB! Indexing in Graph and Document (nosql) databases. 11. May: Vassili.

NB! Last day for the practical work presentations.

  • About the exam.

18. May: guest lecture

Kalle Tomingas. Impact analysis and ontologies.

http://demo.dlineage.com


teadmised_note

Pärit leheküljelt "http://www.lambda.ee/w/index.php?title=Teadmiste_otsing,_formaliseerimine_ja_hoidmine&oldid=15265"