Teadmiste formaliseerimine: erinevus redaktsioonide vahel

Allikas: Lambda
Mine navigeerimisribale Mine otsikasti
← Vanem muudatus
 
(ei näidata 3 kasutaja 440 vahepealset redaktsiooni)
8. rida: 8. rida:
Code: <b>ITI8700</b><br>
Code: <b>ITI8700</b><br>
Link: http://lambda.ee/Teadmiste_formaliseerimine <br>
Link: http://lambda.ee/Teadmiste_formaliseerimine <br>
Lecturer: Tanel Tammet<br>
Lecturer: [https://www.facebook.com/tanel.tammet Tanel Tammet] tanel.tammet@ttu.ee ICT-426<br>
Practice sessions: Riina Maigre<br>
Practice sessions: Riina Maigre<br>
Contact: tanel.tammet@ttu.ee, 6203457, TTÜ ICT-426<br>
Archives of previous years: [[Teadmiste formaliseerimine 2025|2025]].
Archives of previous years: [[Teadmiste formaliseerimine 2023|2023]].
</td>
</td>
</tr>
</tr>
</table>
</table>


= NB! Esimene praktikum on nüüd allpool üleval! =
= Eksamitulemused =


Väikese hilinemisega, aga lõpuks saime praktikumi kokku pandud. Palun tutvu allpool praktikumide osas: käime selle nüüd teisipäeval läbi ka.
Eksamitulemused siin:
* [[Esimene (27 mai) teadmiste eksam 2026]]
* [[Teine  (1 juuni) teadmiste eksam 2026]]
* [[Kolmas  (8 juuni) teadmiste eksam 2026]]


Mis siin varem kirjas oli:
= Eksamikonsultatsioon =


Midagi läks esialgu viltu teamsi grupiga: kogu info kadus ootamatult ära.
(a) väike salvestuse teamsi, teemaks siis eksamikorraldus, ülesanded jne:[https://livettu.sharepoint.com/:v:/r/sites/Knowledgerepresentation2026/Shared%20Documents/General/Recordings/Eksami%C3%BClevaate%20salvestus-20260520_053318UTC-Meeting%20Recording.mp4?csf=1&web=1&e=rQObCH&nav=eyJyZWZlcnJhbEluZm8iOnsicmVmZXJyYWxBcHAiOiJTdHJlYW1XZWJBcHAiLCJyZWZlcnJhbFZpZXciOiJTaGFyZURpYWxvZy1MaW5rIiwicmVmZXJyYWxBcHBQbGF0Zm9ybSI6IldlYiIsInJlZmVycmFsTW9kZSI6InZpZXcifX0%3D salvestus siin], asub kursuse teamsi-salvestuste kataloogis.


Proovime uuesti kursuse 2024 aasta teamsi regada: [https://teams.microsoft.com/l/channel/19%3aEbUrkeGqLsD_szZRLeQlOhnkUIqF84DeefpatHwTQfE1%40thread.tacv2/General?groupId=48c53de1-721e-4fb6-8eb9-e4ccd8300fc5&tenantId=3efd4d88-9b88-4fc9-b6c0-c7ca50f1db57 teamsi channeli link]
(b) Konsultatsioonide lingid siin:  


= Weekly task: for the week 6 Feb to 13 Feb =
* [https://teams.microsoft.com/meet/313481263783182?p=n6Lqqc5BNDUaCUX0IJ eksamikonsultatsioon] 20. mai kell 10:30-11:00. (enne seda, kell 10:00 hajussüsteemide konsultatsioon, tolle kursuse lehel oleval lingil).
* [https://teams.microsoft.com/meet/365383768774501?p=WPjQ5sq3VKweu8doJT eksamikonsultatsioon] 25. mai kell 10:30-11:00. (enne seda, kell 10:00 hajussüsteemide konsultatsioon, tolle kursuse lehel oleval lingil).


Teise nädala ülesanne (<font color="red">6. veebruari</font> loengu järel) on eksperimenteerida veidi [http://logictools.org logictools.org] saidiga ja teha ise läbi loengus toodud näited SQL data ja päringute esitamisest loogikas. Tähtaeg 13. veebruar.
= Eksamikorraldus ja materjalid õppimiseks =


Konkreetselt tee järgmist:
Eksamiks on kokku aega 2.5 tundi, aga enamik peaks saama hakkama ca 1.5 tunniga.
* Eksperimenteeri veidi [http://logictools.org logictools.org] "simple examples" valikuboksi näidetega basic ... equality, loe nende selgitusi ja püüa aru saada tõestustest.
 
* Loe seejuures predikaatloogika väikest seletust [http://logictools.org/about.html#basic_syntax about lehel] kuni lausearvutuseni (seda pole enam otse vaja lugeda).
Eksam on kirjalik, samas ruumis/ajal toimub ka hajussüsteemide kirjalik eksam.
* Kirjuta ise valmis näide vastavalt kursuse materjali [[Media:Kr_lect_1b_2024.pdf| Background: relations of sql and logic]] lk pealkirjaga "SQL join as logical rule", kus on toodud ühe client tabeliga ja client ning cars tabelitega näide sql päringute teisendamisest loogikasse (üks lihtne select ja üks join). Tee need näited reaalselt valmis nii, et logictools.org sööb nad sisse ja annab ootuspärase vastuse. Seejuures on sul vaja kasutada $ans predikaati, nagu "answers" näites toodud. Kui tahad mitut vastust (ei ole kohustuslik), siis vaata ka "multiple answers" näidet.
 
* Saada näite-sisendid ja genereeritud tõestused õppejõule emailiga (riina.maigre at taltech.ee ja cc tanel.tammet at taltech.ee). Kui jääd nendega hätta, siis kirjuta, mida proovisid, too oma näitetekst ja selgita, mille juures hätta jäid (see on ka arvestatav tulemus): kirjuta aadressile riina.maigre at taltech.ee või tanel.tammet at taltech.ee ja pane pealkirja sisse mh sõna "formaliseerimine", et kirja kergesti üles leiaks.
Allolevad punase E1-ga on eksami jaoks kõige olulisemad materjalid, E2 veidi vähem olulised.
NB! Võrreldes varasemaga on lisaks markeeritud E2-ga need varasemalt olemas, aga markeerimata materjalid:
 
* [[Media:Llm_intro_2026.pdf|Lecture material part 1]] <font color="red">E2</font>
* [[Media:Llm_details_2026.pdf|Lecture material part 2]] <font color="red">E2</font>
* [[Media:Llm_usage_rag.pdf|Lecture material]] <font color="red">E2</font>
* [[Media:Visual Reasoning 2026.pdf|Lecture presentation]] <font color="red">E2</font>
 
Ekamil tuleb neli eraldi hinnatavat osa: kolm teemaküsimust ja üks osa paari lühiküsimusega. Spetsiifiliste süntaksite (RDF, loogikavarandid jne) osas: ühtegi täpset süntaksit/võtmesõnu ei pea teadma, aga esitus peab olema õppejõule sisuliselt arusaadav.
Millised on teemaküsimused ja millised on lühiküsimused:
 
SQL ja loogika: esitan paar tabelit ja küsin mõnda taolist küsimust:
* nende sisu võimalikult täpset esitust esimest järku loogikas või RDFs
* mõne konkreetse where- või join-päringu kirjutamist esimest järku loogikas
 
JSON-LD: annan JSON-LD jupi ja küsin selle tähendust ehk teisendust RDF kujule kolmikutena.
Reeglite esitamine: pane rdf ja/või rdfs kujule ja/või predikaatloogikale baseerudes kirja etteantud konkreetsed inimkeelsed väited.
 
Reeglite rakendamine klassikalises teoreemitõestamises: oskus teha käsitsi läbi väike resolutsioonimeetodi rakendamise näide, kui disjunktid ette antud.
 
Sõnavektorid ja N-grammid ja keelemudelite üldprintsiibid: arusaamine, oskus esitada väike näide töötamise kohta, võimalikud kasutus- või loomisstsenaariumd
 
Ebakindlate teadmiste esitamise põhimeetodid: pidevad (erinevat sorti tõenäosuste-tüüpi esitus) ja diskreetsed (eranditega süsteemid, eeskätt default logic). Siin võin tuua väikese näite ja küsida lahendust või palun ise esitada mõne pisinäite sobival kujul. Konkreetseid süntakseid teadma ei pea, piisab õppejõule arusaadavaks tegemisest.
Lühiküsimused (siit valitakse iga eksami jaoks paar):
 
Seleta paari lausega, mis asi on, vahel küsin ka näidet: freimiprobleem, open/closed world, blocks world, RAG, erinevad tõenäosuste interpretatsioonid, RDFa; mis asi on ja mida sealt umbes leida võib ja mida pigem mitte: schema.org, wordnet, conceptnet, yago, cyc; mida sa mäletad "semantiline parsimine" ja "visual reasoning" teemade kohta.
 
= Eksamipäevad ja ajad =
 
Siin on täpsustunud esimese kahe eksami ajad:
 
* 27 mai, kolmapäev, kell 11:00 (koos hajussüsteemide eksamiga). Auditoorium NRG-226.
* 1 juuni, esmaspäev: eraldi eksamiajad hajussüsteemide (11:00) ja teadmiste esitamise (14:00) jaoks. Auditoorium SOC-211a.
* 8 juuni, esmaspäev: kell 11:00 (koos hajussüsteemide eksamiga). Auditoorium ICT-315.
 
 
<!--
= <font color="red">NB! Kolmas kodutöö uuendatud</font> =


Soovitused:
Vaata allpool "Third lab: questions in NLP": see on uuendatud 2026 aasta jaoks. Idee on analoogiline eelmise aasta ülesandega, aga konkretiseeritud piltide ülesandega (esimese ja teise kodutöö teemad) sobivaks.
* Negatiivset arvu sa logictoolsi vaikimisi süntaksiga otse sisestada ei saa: -200 interpreteeritakse kui loogilise eituse rakendamist 200-le. Kaks head võimalust negatiivse arvu sisestamiseks on kas kirjutada 0-200 või $difference(0,200).
-->
* Aritmeetika jaoks vaata help teksti Arithmetic peatükki.
<!--
* Väiksem-tingimuse jaoks vaata complex examples valikuboksis "arithmetic" näidet.
= <font color="red">NB! 23 ja 24 veebruar on puhkepäevad</font> =
Esmaspäev, 23. veebruar (ja ka teisipäev, 24. veebruar) on ülikoolis puhkepäevad ja õppetööd ei toimu.
-->
<!--
= <font color="red">NB! Materjalid hetkel uuendamisel</font> =
-->


= Time, place, result =
= Time, place, result =


Lectures: Tuesday 12:00-13:15 room ICT-315 <br>  
Lectures (Tanel): Mondays 12:00-13:30 room SOC 414<br>  
Practical work: Tuesday 14:00-17:15 room ICT-315: every second week (1,3,5, etc, starting with 30. Jan, then 13 Feb, then 27 Feb etc)
Practice sessions (Riina): Tuesdays 08:00-09:30 room <s>MEK-122</s> MEK-109.


See oli esialgne [https://teams.microsoft.com/l/team/19%3aEbUrkeGqLsD_szZRLeQlOhnkUIqF84DeefpatHwTQfE1%40thread.tacv2/conversations?groupId=48c53de1-721e-4fb6-8eb9-e4ccd8300fc5&tenantId=3efd4d88-9b88-4fc9-b6c0-c7ca50f1db57 Teamsi link] millega läks midagi viltu (kogu info kadus äkki ära). Palun proovi nüüd ennast sinna uuesti regada! Igaks juhuks ka [https://teams.microsoft.com/l/channel/19%3aEbUrkeGqLsD_szZRLeQlOhnkUIqF84DeefpatHwTQfE1%40thread.tacv2/General?groupId=48c53de1-721e-4fb6-8eb9-e4ccd8300fc5&tenantId=3efd4d88-9b88-4fc9-b6c0-c7ca50f1db57 channeli link], kui see peaks paremini töötama.  
The lectures and sessions will be on site (physical room) plus teams. Later we may decide to do some lectures/practice sessions on teams only: this will be advertised on the page here early. Beware: recording may not always succeed.


Plaan on teha osa loengutest kohapeal (esialgu kindlasti kohapeal) ja osa distantsilt. Loengutest teeme teamsi ülekande, salvestame ja salvestuste lingid on edaspidi allpool blokkide/loengute juures. Esimene salvestus on [https://livettu.sharepoint.com/:v:/r/sites/Knowledgerepresentation2023/Shared%20Documents/General/Recordings/Meeting%20in%20_General_-20240206_120353-Meeting%20Recording.mp4?csf=1&web=1&e=VZgTGB&nav=eyJyZWZlcnJhbEluZm8iOnsicmVmZXJyYWxBcHAiOiJTdHJlYW1XZWJBcHAiLCJyZWZlcnJhbFZpZXciOiJTaGFyZURpYWxvZy1MaW5rIiwicmVmZXJyYWxBcHBQbGF0Zm9ybSI6IldlYiIsInJlZmVycmFsTW9kZSI6InZpZXcifX0%3D 6. veebruarist] (ta läks kogemata 2023 teamsi, aga peaks olema ikka kenasti näha).
Please join the [https://teams.microsoft.com/l/team/19%3A-xXLRsURTirXcCCkQN46w4wYRJaarxbl797KeThtWXY1%40thread.tacv2/conversations?groupId=30b97413-f1b1-404d-8498-b35024d1a002&tenantId=3efd4d88-9b88-4fc9-b6c0-c7ca50f1db57 teams group] of the course. The code for joining is zb3z457.


Weekly [https://teams.microsoft.com/meet/33688408323196?p=sd6YBPK9VZ7gRe7Zro teams link for the lecture].


Siit saad näha salvestust [https://echo360.org.uk/section/30408d7b-4704-4898-a478-12031ea7cc7a/home esimese loengu kohta], mis ei olnud teamsis.
Weekly [https://teams.microsoft.com/meet/3558703223396?p=zc4Lgecwoy1e9Yhav9 teams link for the practice session].


Praksidega samamoodi: algul kindlasti kohapeal, samas üritame salvestada / üle kanda, edaspidi tõenäoliselt osa prakse distantsilt (teamsis).
= Grading =


Salvestuste lingid leiad allpool kas vastava loengu või praksi peatüki all.
Practical work will give 50%, exam 50% of points, and weekly tasks additional extra up to 10%, underlying the final grade. The exam will consist of (a) several questions asking for the explanation of important concepts along with examples (b) some small excercises.  


Moodles materjale ei ole: kogu info, materjalid ja ülesanded on lambda lehel.  
The materials below marked with the red <font color="red">E1</font> are the main materials for the exam (10 of these on the page), while <font color="red">E2</font> materials (20 on the page) will also come up, but less and/or in a simpler form.  


Practical work will give 50%, exam 50% of points, and weekly tasks additional extra up to 10%, underlying the final grade. The exam will consist of several small excercises.
To finish the course successfully, you need to


To finish the course successfully, you need to
* successfully pass three practical works
* successfully pass three practical works
* receive at least 1/3 of the max points at exam
* receive at least 1/3 of the max points at exam
71. rida: 118. rida:


Hence the main focus of the course is on hybrid methods for  
Hence the main focus of the course is on hybrid methods for  
[https://en.wikipedia.org/wiki/Knowledge_representation knowledge representation and reasoning]: [https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence symbolic AI], [https://en.wikipedia.org/wiki/Machine_learning machine learning / neural methods] and their [https://en.wikipedia.org/wiki/Neuro-symbolic_AI combinations].  
[https://en.wikipedia.org/wiki/Knowledge_representation knowledge representation and reasoning]: [https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence symbolic AI], [https://en.wikipedia.org/wiki/Machine_learning machine learning / neural methods] and their [https://en.wikipedia.org/wiki/Neuro-symbolic_AI neurosymbolic combinations].  
We look at this spectrum from simple (representing and using knowledge in databases) to complex (meaning of sentences in natural language and commonsense knowledge) tasks.
We look at this spectrum from simple (representing and using knowledge in databases) to complex (meaning of sentences in natural language and commonsense knowledge) tasks.
A closely related subject is [https://en.wikipedia.org/wiki/Commonsense_reasoning commonsense reasoning].
A closely related subject is [https://en.wikipedia.org/wiki/Commonsense_reasoning commonsense reasoning].
83. rida: 130. rida:
* Uncertain knowledge and hybrid systems
* Uncertain knowledge and hybrid systems


<!--
= Weekly tasks =
Check out [http://www.lisperati.com/tellstuff/ this description] of the whole KR area.


See also the intro lecture [[Media:Knowledge_Representation_Intro.pptx]]
The small weekly tasks are in addition to the practical work: their goal is to practice with the stuff presented at the last lecture.  
-->


= Links to the teams group and lecture videos =
== The first weekly task ==
<!--
Please join the [https://teams.microsoft.com/l/team/19%3aB8Z1aw0MCoJ2RiUXWWjUK0PAj77TuMKKdotlw5Vfnhw1%40thread.tacv2/conversations?groupId=480b7938-a312-47e7-bad0-5ea1bb51f31e&tenantId=3efd4d88-9b88-4fc9-b6c0-c7ca50f1db57  teams site for the course]
-->


The links to saved lecture videos are and will be  visible among the materials of corresponding lectures below.
Ülesanne on eksperimenteerida veidi [http://logictools.org logictools.org] saidiga ja teha ise läbi loengus toodud näited SQL data ja päringute esitamisest loogikas. Tähtaeg 17. veebruar.
<!--
The second lecture at 4. Feb at 14:00 will be in Teams
[https://teams.microsoft.com/l/meetup-join/19%3ac4d77289f3c84a43bab641e5f9ce8284%40thread.tacv2/1612439265230?context=%7b%22Tid%22%3a%223efd4d88-9b88-4fc9-b6c0-c7ca50f1db57%22%2c%22Oid%22%3a%22e3e55dca-e89b-46ae-b8ee-1d158e79281d%22%7d at this link].  
-->


<font color="red">NB!</font> In case you run into trouble (cannot join or no sound etc) please write via [https://www.facebook.com/tanel.tammet/ fb messenger] or email tanel.tammet at taltech.ee.
Konkreetselt tee järgmist:
<!--
* Eksperimenteeri veidi [http://logictools.org logictools.org] "simple examples" valikuboksi näidetega basic ... equality, loe nende selgitusi ja püüa aru saada tõestustest.  
There is also a [https://moodle.taltech.ee/course/view.php?id=31546 site for grades] registration code 4bFHBTrq. For some time there will be no grades there. -->
* Loe seejuures predikaatloogika väikest seletust [http://logictools.org/about.html#basic_syntax about lehel] kuni lausearvutuseni (seda pole enam otse vaja lugeda).  
<!--
* Kirjuta ise valmis näide vastavalt kursuse materjalile [[Media:Kr_lect_1b_2024.pdf| Background: relations of sql and logic]] lk pealkirjaga "SQL join as logical rule", kus on toodud ühe client tabeliga ja client ning cars tabelitega näide sql päringute teisendamisest loogikasse (üks lihtne select ja üks join):
The first lecture should be visible [https://web.microsoftstream.com/video/a97786c8-11db-4c9b-a86b-c338a059a6e4 here].
<pre>
-->
client table: id name balance
1 john 100
2 pete -200


<!--
cars table: id model owner
= Weekly tasks =
1 ford 1
2 opel 2
3 saab 2


Teine nädalaülesanne (15. märtsi loengu järel) on katsetada ise sõnavektorite loomist. Selleks
select client.name, client.balance
* Võta mingi hulk pikki tekstifaile. Võid võtta ka terve [http://turing.cs.ttu.ee/~tanel.tammet/wikicompact.tar.gz lemmatiseeritud wikipedia] ja valida sealt välja näiteks mõne alamkataloogi, aga võid võtta ka mõne muu pikema tekstikorpuse.
from client where balance<0
* Vali mingi hulk N (näiteks 100) representatiivset sõna (vastavad vektori tulpadele).
* Tee programm, mis käib tekstifailid läbi ja koostab iga sealt leitud sõna kohta massiivi, kus igas tulbas on co-occurrencite koguarv.
** Co-occurrenceid vaata näiteks 4 sõna vasakule ja 4 sõna paremale. Näiteks, kui leitud sõna on "car" ja tekstijupp on "cool red car was driving" ja "red" ning "driving" on representatiivsete sõnade hulgas, siis "car" jaoks suurendame "red" ja "driving" tulpade sisu ühe võrra.
** Kokkuvõttes saad tabeli, kus iga rida vastab ühele leitud sõnale ja iga tulp vastab selle sõna co-occurrencite koguarvule tulbale vastavate representatiivsete sõnade jaoks.
* Sorteeri tabeli read nii, et mingis mõttes enimesinenud sõnad oleks eespool (näiteks, loe iga sõna jaoks kokku tulpades olevad numbrid).
* Uuri veidi tabelit: mida huvitavat leiad?
* Kui oled veidi ambitsioonikam, siis püüa koostada sõnade sarnasuse hindamise programm, mis kasutab koosinus-sarnasust. Seda saab teha mitmel moel: uuri [https://web.stanford.edu/~jurafsky/slp3/6.pdf õpiku peatükist].
-->
<!--
Esimene nädalaülesanne (15. märtsi loengu järel) on katsetada lihtsa json-ld ja rdfs reeglite kasutamist päringu jaoks. Tähtaeg <font color="red">28</font>. veebruari õhtu: kui vahepeal hätta jääd, küsi emailitsi või loengus abi.


Konkreetselt on sul antud kaks json-ld teksti
select client.name, cars.model from client, cars
<pre>
where client.id=cars.owner
{
  "@id":"https://et.wikipedia.org/wiki/Peipsi_j%C3%A4rv",
  "@type": "lake",
  "size": "large",
  "country": "Estonia"
}
{
  "@id":"https://en.wikipedia.org/wiki/Snelli_Pond",
  "@type": "pond",
  "country": "Estonia"
} 
</pre>
ja kolm rdf stiilis kolmikut
<pre>
<lake>, <rdfs:subclassOf>, <body_of_water>
<pond>, <rdfs:subclassOf>, <body_of_water>
<pond>, <size>, <small>
</pre>
ja loomulikus keeles reegel
<pre>
Water skiing is possible on large bodys of water.
</pre>
</pre>


Sinu ülesanne on teisendada kõik need tavaloogikasse ja teha https://logictools.org abil kaks eksperimenti:
Tee need näited reaalselt valmis nii, et logictools.org sööb nad sisse ja annab ootuspärase vastuse. Seejuures on sul vaja kasutada $ans predikaati, nagu "answers" näites toodud. Kui tahad mitut vastust (ei ole kohustuslik), siis vaata ka "multiple answers" näidet.
* Näita, et Peipsi järvel saab veesuusatada.
 
* Otsi $ans predikaadi abil kohad Eestis, kus saab veesuusatada (Peipsi peaks vastusesse tulema ja Snelli tiik mitte).
* Saada näite-sisendid ja genereeritud tõestused õppejõule emailiga (riina.maigre at taltech.ee ja cc tanel.tammet at taltech.ee). Kui jääd nendega hätta, siis kirjuta, mida proovisid, too oma näitetekst ja selgita, mille juures hätta jäid (see on ka arvestatav tulemus): kirjuta aadressile riina.maigre at taltech.ee või tanel.tammet at taltech.ee ja pane pealkirja sisse mh sõna "formaliseerimine", et kirja kergesti üles leiaks.
 
Seejuures pead mõtlema,  
* Kasuta kindlasti oma taltech.ee aadressi, sest muidu võib kiri lõpetada rämpspostis!
* kuidas kirjutada reegel rdf:type ja rdfs:subclassOf sisulise koos-kasutamise jaoks.  
* kuidas oleks mõistlik esitada loogikas reegel "Water skiing is possible in large bodys of water": kuna seda otse rdfs vahenditega kirjutada ei saa, kirjuta ta lihtsalt tavaloogikas.
* optsionaalselt: kuidas teha reegel, et  pond, size, small oleks kasutatav. Otseselt seda ülesandes vaja ei lähe.


Soovitused:
Soovitused:
* Väldi namespace-sid ja liigpikki identifikaatoreid, neid ei ole siin ülesandes tegelikult vaja.  
* Negatiivset arvu sa logictoolsi vaikimisi süntaksiga otse sisestada ei saa: -200 interpreteeritakse kui loogilise eituse rakendamist 200-le. Kaks head võimalust negatiivse arvu sisestamiseks on kas kirjutada 0-200 või $difference(0,200).
* Käsitle "water skiing" kui ühte sõna, a la "water_skiing", samuti "is possible on" kui "is_possible_on".
* Aritmeetika jaoks vaata help teksti Arithmetic peatükki.
* Väiksem-tingimuse jaoks vaata complex examples valikuboksis "arithmetic" näidet.
 
 
== The second weekly task ==
 
Selle ülesande mõte on veidi praktiseerida JSON-LD esitusega. Sinu konkreetne ülesanne on võtta oma esimene nädalaülesanne ja esitada needsamad andmed (ja soovitavalt ka reeglid) JSON-LD kujul.
 
Vaata alustuseks json-ld näiteid [https://logictools.org/json.html logictoolsi json-tabi alt]. NB! Enamus näiteid "example" valikuboksist ei ole konventsionaalne json-ld, aga mitmed on.
 
<!--
= The first weekly task =


Kui see kõik lihtsalt välja tuleb, võid vabatahtlikult katsetada täiendavat ülesannet: mida tuleks lisada, et õnnestuks näidata, et Snelli tiigil ei saa veesuusatada.
Ülesanne on eksperimenteerida veidi [http://logictools.org logictools.org] saidiga ja teha ise läbi loengus toodud näited SQL data ja päringute esitamisest loogikas. Tähtaeg 25. veebruar.


Emaili andmete esitus ja leitud tõestused aadressile tanel.tammet at taltech.ee, pannes emaili pealkirja sisse sõna "formaliseerimine".
Konkreetselt tee järgmist:
* Eksperimenteeri veidi [http://logictools.org logictools.org] "simple examples" valikuboksi näidetega basic ... equality, loe nende selgitusi ja püüa aru saada tõestustest.
* Loe seejuures predikaatloogika väikest seletust [http://logictools.org/about.html#basic_syntax about lehel] kuni lausearvutuseni (seda pole enam otse vaja lugeda).
* Kirjuta ise valmis näide vastavalt kursuse materjali [[Media:Kr_lect_1b_2024.pdf| Background: relations of sql and logic]] lk pealkirjaga "SQL join as logical rule", kus on toodud ühe client tabeliga ja client ning cars tabelitega näide sql päringute teisendamisest loogikasse (üks lihtne select ja üks join). Tee need näited reaalselt valmis nii, et logictools.org sööb nad sisse ja annab ootuspärase vastuse. Seejuures on sul vaja kasutada $ans predikaati, nagu "answers" näites toodud. Kui tahad mitut vastust (ei ole kohustuslik), siis vaata ka "multiple answers" näidet.
* Saada näite-sisendid ja genereeritud tõestused õppejõule emailiga (riina.maigre at taltech.ee ja cc tanel.tammet at taltech.ee). Kui jääd nendega hätta, siis kirjuta, mida proovisid, too oma näitetekst ja selgita, mille juures hätta jäid (see on ka arvestatav tulemus): kirjuta aadressile riina.maigre at taltech.ee või tanel.tammet at taltech.ee ja pane pealkirja sisse mh sõna "formaliseerimine", et kirja kergesti üles leiaks.
* Kasuta saatmiseks kindlasti taltech.ee aadressi, sest muidu võib kiri minna rämpspostiks!
 
Soovitused:
* Negatiivset arvu sa logictoolsi vaikimisi süntaksiga otse sisestada ei saa: -200 interpreteeritakse kui loogilise eituse rakendamist 200-le. Kaks head võimalust negatiivse arvu sisestamiseks on kas kirjutada 0-200 või $difference(0,200).
* Aritmeetika jaoks vaata help teksti Arithmetic peatükki.
* Väiksem-tingimuse jaoks vaata complex examples valikuboksis "arithmetic" näidet.
-->
-->


= Practical work =
= Practical work =


Actual 2024 stuff finally here!


There are three labs.
There are three labs.
<!-- They are all steps in a single project: build a simple natural-language question-answering system a la [https://en.wikipedia.org/wiki/Watson_(computer) classic Watson]. There is also an alternative option to do one larger practical research project instead of these three labs: details of this project have to be discussed and agreed with Tanel.
 
-->
The labs have to be presented (as a demo + small overview of the software and principles) to the course teachers and all students present at labwork time, either in the auditorium or over MS teams. The labs can be prepared alone or by teams of two or three people.  
The labs have to be presented (as a demo + small overview of the software and principles) to the course teachers and all students present at labwork time, either in the auditorium or over MS teams. The labs can be prepared alone or by teams of two or three people.  


<!--
First, read the explanation of the [[KR 2019 homework: overall task|overall task for the 2020 course]] [[KR 2019 homework: overall task|overall task for the recent 2019 course]].  The following labs are steps on the path to the end goal.
-->
Each lab is evaluated independently and gives the same number of points.  
Each lab is evaluated independently and gives the same number of points.  
NB! When you do not meet the lab deadline, you will be awarded half of the points for this lab.
NB! When you do not meet the lab deadline, you will be awarded half of the points for this lab.
<!--
[[notes for kr praks 2024]]
-->


== First lab ==
== First lab ==


Updated for 2024!
The [[KR 2026 homework part 1|2026 task for the first lab]] is about encoding nontrivial information in various ways (standard logic, rdf, json-ld) and writing simple logical rules for answering questions.


The [[KR 2024 homework part 1|2024 task for the first lab]] is about handling and understanding triplet stores, meaning of rdf types/properties/etc and writing simple logical rules for answering questions.
Deadline: 10. March.
 
<!--
You [[KR 2023 homework part 1|2023 task for the first lab]] is an intro to the world of large knowledge bases and a preparation for the following labs where we will use this data to experiment with some nontrivial question answering.
-->
Deadline: 28 February.


== Second lab ==
== Second lab ==


Not updated for 2024 yet. These were tasks for 2023:
The [[KR 2026 homework part 2|2026 task for the second lab]] is about using a VLM (visual language model) to get a structured description from an image which would fit with the pre-existing simple world model.


Deadline: 11. April 2022.
Deadline: 7. April
 
See [[Details of the KR second lab in 2023]]


== Third lab: questions in NLP ==
== Third lab: questions in NLP ==


Deadline: 9. May.
This is the task for 2026:


Not updated for 2024 yet. These were tasks for 2023:
[[Experiment with neurosymbolic reasoning: LLM as a parser plus a symbolic reasoner for images: 2026]]


* [[Experiment with natural language rules, ChatGPT and ProofWriter]] (less technical): this gives 2/3 of full points.
Deadline: 12. May
* [[Use a semantic parser and a reasoner to solve questions posed in natural language]] (more technical): this gives full points.
 
Details under the separate pages linked to above. An intro and initial explanations will be given during the lectures, starting with the 28 March lecture.
 
Since the tasks are conceptually nontrivial, everybody is encouraged to use the [https://teams.microsoft.com/_#/school/conversations/General?threadId=19:pbCAi1dcW-0ZYLt9oaEWEF0OVSCh_itNLauu6apjLII1@thread.tacv2&ctx=channel teams group] for the course to ask for help whenever you run into trouble: starting from April both teachers will monitor the teams daily and try to help you out.  


<!--
<!--
== Alternative to labs: a research project ==
This is an example from 2025, not the task for 2026:
 
[[Experiment with neurosymbolic reasoning: LLM as a parser plus a symbolic reasoner: 2025]]
The goal of the research project is to build a good database of similar words along with the main differences between word/similar word. A la "queen" is similar to "king" except for gender etc. There are a number of papers explaining different methods. Tanel has built a prototype database/software for doing this from Wikipedia texts + NLP tools + Wordnet, but this prototype is too weak and also does not find main differences. The prototype can be used as a starting point.
-->
<!-- * [[Use a semantic parser and a reasoner to solve questions posed in natural language: 2025]] -->
<!-- We will do live experimenting for the third lab at the April 22 lecture and will then add more interesting parts to the notes above. -->
<!--
NB! The task description above has just been updated (at 13. April) with the example code and files developed and presented during the [https://livettu.sharepoint.com/:v:/r/sites/Knowledgerepresentation2025/Shared%20Documents/General/Recordings/Knowledge%20representation%202025%20lectures-20250422_140202-Meeting%20Recording.mp4?csf=1&web=1&e=McaLE9&nav=eyJyZWZlcnJhbEluZm8iOnsicmVmZXJyYWxBcHAiOiJTdHJlYW1XZWJBcHAiLCJyZWZlcnJhbFZpZXciOiJTaGFyZURpYWxvZy1MaW5rIiwicmVmZXJyYWxBcHBQbGF0Zm9ybSI6IldlYiIsInJlZmVycmFsTW9kZSI6InZpZXcifX0%3D 22. April lecture.]
-->
-->


228. rida: 240. rida:


* Get the basic background about classical automated reasoning from the [[Media:Loogikaopik.pdf|book chapter pages 297-345]] by Tanel Tammet and optionally more details from  [http://lambda.ee/w/images/0/06/Geoffreasoningnotes.pdf the set of course notes] from Geoff Sutcliffe.
* Get the basic background about classical automated reasoning from the [[Media:Loogikaopik.pdf|book chapter pages 297-345]] by Tanel Tammet and optionally more details from  [http://lambda.ee/w/images/0/06/Geoffreasoningnotes.pdf the set of course notes] from Geoff Sutcliffe.
* <b>The</b> book for natural language (NLP): Speech and Language Processing by Dan Jurafsky and James H. Martin. [https://github.com/rain1024/slp2-pdf/blob/master/complete-book-pdf/slp2.pdf here is the 2nd edition] and (suggested) [https://web.stanford.edu/~jurafsky/slp3/ here the web page for the draft 3rd edition]. Recommendation: read from the beginning up to an including chapter 6: Vector Semantics and Embeddings, then skip and read a few chapters starting from 15: Logical Representations of Sentence Meaning. You do not need to understand all the details of math, for example. It is OK if you understand the main principles of representations and algorithms.
* <b>The</b> book for natural language (NLP): Speech and Language Processing by Dan Jurafsky and James H. Martin. <!-- [https://github.com/rain1024/slp2-pdf/blob/master/complete-book-pdf/slp2.pdf here is the 2nd edition] and (suggested) --> [https://web.stanford.edu/~jurafsky/slp3/ the web page for the draft 3rd edition]. Recommendation: read from the beginning up to an including chapter 6: Vector Semantics and Embeddings, then read [https://web.stanford.edu/~jurafsky/slp3/20.pdf chapter 6 about extracting information] and the appendix chapter [https://web.stanford.edu/~jurafsky/slp3/H.pdf Logical Representations of Sentence Meaning].


Other useful books and tools for symbolic K&R:
Other useful books and tools for symbolic K&R:


* [https://logictools.org logictools.org] for experimenting with classical automated reasoning
* [https://logictools.org logictools.org] for experimenting with classical automated reasoning
* Symbolic reasoning themes are covered in [http://www.sciencedirect.com/science/book/9781558609327 this book] with freely accessible pdf-s.
* Symbolic reasoning themes are covered in [http://www.sciencedirect.com/science/book/9781558609327 this book] with (hopefully) freely accessible pdf-s via Taltech. See also [https://www.cs.toronto.edu/~hector/PublicKRSlides.pdf slides] for the book.
* Similarly, the freely accessible pdf of the [http://ii.fmph.uniba.sk/kri/KRhandbook.pdf handbook of knowledge representation] gives a detailed and thorough coverage of the subject; far more than necessary for the course.
* Similarly, the freely accessible pdf of the [https://dai.fmph.uniba.sk/~sefranek/kri/handbook/handbook_of_kr.pdf handbook of knowledge representation] <!-- http://ii.fmph.uniba.sk/kri/KRhandbook.pdf  --> gives a detailed and thorough coverage of the subject; far more than necessary for the course.
* You may want to get a small intro to Prolog by reading the first 100 pages of a classic [https://mitpress.mit.edu/books/art-prolog-second-edition Art of Prolog] book: pdf is freely downloadable under the "Open Access" tab. It is not strictly necessary for the course (we will not be using Prolog) but it surely helps. Besides, the book is interesting, quite easy to read and works out a lot of interesting examples.
* You may want to get a small intro to Prolog by reading the first 100 pages of a classic [http://cdn.preterhuman.net/texts/computing/programming/Prolog/The%20Art%20of%20Prolog%20%202nd%20Ed%20-%20Leon%20Sterling%20,%20Ehud%20Shapiro.pdf Art of Prolog] book. It is not strictly necessary for the course (we will not be using Prolog) but it surely helps. Besides, the book is interesting, quite easy to read and works out a lot of interesting examples.


Other useful material for neural NLP:
Other useful material for neural NLP:


* Andrej Karpathy [https://github.com/karpathy/nanoGPT nanoGPT] and [https://github.com/karpathy/nn-zero-to-hero neural networks zero to hero course]
* Andrej Karpathy [https://github.com/karpathy/nanoGPT nanoGPT] and [https://github.com/karpathy/nn-zero-to-hero neural networks zero to hero course]
* [https://dugas.ch/artificial_curiosity/GPT_architecture.html GPT architecture] and [https://drive.google.com/file/d/1UOfN0iB_A0rEGYc2CbYnpIF44FupQn2I/view chatgpt cheatsheet]
<!-- * [https://dugas.ch/artificial_curiosity/GPT_architecture.html GPT architecture] -->
 
 
 
<!--
<!--
* Interesting to browse: [http://www.aaai.org/Library/KR/kr12contents.php recent conference proceedings]
* Interesting to browse: [http://www.aaai.org/Library/KR/kr12contents.php recent conference proceedings]
253. rida: 262. rida:
= Blocks and lectures: the main content of the course =
= Blocks and lectures: the main content of the course =


Allpool olev blokkide struktuur on sarnane, aga mitte sama, kui 2022 aastal. Transformerite (BERT, GPT) ja hübriidsüsteemide osa muutub mahukamaks.
The details and materials of lectures in the future are from 2025 and will be revised/extended. The materials of the current or passed lectures are up to date.


Toimunud loengute materjalid ja salvestused osaliselt uuendatakse/muudetakse: kogu info veel mitte toimunud loengute ja prakside kohta on pärit peamiselt 2022 aastast ja on esialgne indikatsioon.


== Block 1: knowledge in SQL, RDF and JSON ==
== Block 1: knowledge in SQL, RDF and JSON ==


=== Lecture: Intro and background, 30. Jan ===
=== Lecture: Intro and background: 2. February ===


Sissejuhatuse presentatsioon:
Intro presentation:


[[Media:teadmised_intro2024.pdf|teadmised_intro2024.pdf]]
[[Media:teadmised_intro2026.pdf|teadmised_intro2026.pdf]]


Soovitav (mitte kohustuslik) kuulamine:
Recommended non-obligatory listening: [https://lexfridman.com/francois-chollet-2 Francois Chollet], [https://lexfridman.com/yann-lecun-2 Yann LeCun], [https://lexfridman.com/gary-marcus Gary Marcus], [https://lexfridman.com/demis-hassabis Demis Hassabis], [https://lexfridman.com/ilya-sutskever Ilya Sutskever], [https://lexfridman.com/ben-goertzel Ben Goertzel] <!--, [https://towardsdatascience.com/language-models-symbolic-learning-and-the-road-to-agi-75725985cdf7 Tim Rocktäschel] -->,
* Marcus episode on the Lex Fridman podcast https://lexfridman.com/gary-marcus/
* Gary Marcus (KR&R commonsense proponent) & Joshua Bengio (ML proponent) https://www.facebook.com/watch/live/?v=498403850881660&ref=watch_permalink read also the related Marcus paper https://arxiv.org/ftp/arxiv/papers/1801/1801.00631.pdf
* David Ferrucci (IBM Watson team leader) on the Lex Fridman podcast https://lexfridman.com/david-ferrucci/
* Tim Rocktäschel (Facebook A.I.) episode https://towardsdatascience.com/language-models-symbolic-learning-and-the-road-to-agi-75725985cdf7
* Reading, not listening: on Hybrid A.I. in image analysis https://knowablemagazine.org/article/technology/2020/what-is-neurosymbolic-ai


=== Lecture: Nosql, json and rdf in databases: 6. Feb ===
=== Lecture: Nosql, json and rdf in databases: 9. February ===


Lecture [https://livettu.sharepoint.com/:v:/r/sites/Knowledgerepresentation2023/Shared%20Documents/General/Recordings/Meeting%20in%20_General_-20240206_120353-Meeting%20Recording.mp4?csf=1&web=1&e=VZgTGB&nav=eyJyZWZlcnJhbEluZm8iOnsicmVmZXJyYWxBcHAiOiJTdHJlYW1XZWJBcHAiLCJyZWZlcnJhbFZpZXciOiJTaGFyZURpYWxvZy1MaW5rIiwicmVmZXJyYWxBcHBQbGF0Zm9ybSI6IldlYiIsInJlZmVycmFsTW9kZSI6InZpZXcifX0%3D recording in teams].
Lecture materials:
Lecture materials:
* We cover [[Media:Kr_lect_1b_2024.pdf| Background: relations of sql and logic]] <font color="red">E1</font>
* We cover [[Media:Kr_lect_1b_2024.pdf| Background: relations of sql and logic]] <font color="red">E1</font>


* and start with [[Media:Kr_lect_2b_2024.pdf| schemaless databases, RDF and RDFS]] <font color="red">E1</font> (also important for practice work)
* and start with [[Media:Kr_lect_2b_2024.pdf| schemaless databases, RDF and RDFS]] <font color="red">E1</font> (also important for practice work)
<!--
* [https://livettu.sharepoint.com/:v:/r/sites/Knowledgerepresentation2022/Shared%20Documents/General/Recordings/Meeting%20in%20_General_-20220201_100310-Meeting%20Recording.mp4?csf=1&web=1&e=VcZTmK Lecture recording in teams]
-->
<!--
These materials will be taken up in the next lecture:
* [[Nosql links and notes]] <font color="red">E2</font>
* Json in Postgresql [https://www.postgresqltutorial.com/postgresql-json/ intro tutorial] <font color="red">E2</font> and official [https://www.postgresql.org/docs/current/datatype-json.html json datatype] and especially [https://www.postgresql.org/docs/current/functions-json.html functions and operations]
-->
See also
See also
* Json in Postgresql: [https://www.postgresql.org/docs/current/datatype-json.html json datatype] and [https://www.postgresql.org/docs/current/functions-json.html functions for json]
* Json in Postgresql: [https://www.postgresql.org/docs/current/datatype-json.html json datatype] and [https://www.postgresql.org/docs/current/functions-json.html functions for json]
* [[notes about schema for rdf / first lab]]


The weekly homework will be about trying out small things in logic: please try out [http://logictools.org/ logictools] from "basics" to "equality" in simple examples selectbox. Details will appear after the lecture.
The weekly homework will be about trying out small things in logic: please try out [http://logictools.org/ logictools] from "basics" to "equality" in simple examples selectbox. Details will appear after the lecture.
<!--
* [https://json-ld.org/ json-ld]: currently most popular rdf syntax (in json), also recommended by Google.
-->


== Block 2: handling simple facts and rules in symbolic AI and logic ==
== Block 2: handling simple facts and rules in symbolic AI and logic ==




=== Lecture: simple rules in rdfs, logic and owl: 14 feb ===
=== Lecture: simple rules in rdfs, logic and owl + reasoner: 16. February ===
 
 


* Main lecture material: [[Media:Kr_lect_2b.ppt| schemaless databases, RDF and RDFS]] looked at rdfs rule meanings in logic and sparql (up to page 30 in the presentation).
* Main lecture material: [[Media:Kr_lect_2b.ppt| schemaless databases, RDF and RDFS]]
<!--
<!--
* [https://echo360.org.uk/media/39c66a70-0b79-4fb9-b516-affdd1cbe108/public Lecture recording in echo].
* [https://livettu.sharepoint.com/:v:/s/Knowledgerepresentation2023/EflyEjMnTMxKji6nxhy1ErUBvaUg1I749ZjrOtNYrHUynQ?e=cBfYNu 2023 Loengusalvestuse link].
-->
-->
* [https://livettu.sharepoint.com/:v:/s/Knowledgerepresentation2023/EflyEjMnTMxKji6nxhy1ErUBvaUg1I749ZjrOtNYrHUynQ?e=cBfYNu Loengusalvestuse link].
[[Rdf example developed during lecture]]<font color="red">E2</font>
* [https://livettu.sharepoint.com/:v:/s/Knowledgerepresentation2023/EXxSuoyDiidApoXqKs8SAkwBuvebQ7kg3iDeGQOD4_cyoQ?e=A67Grh Praksi salvestuse link].
* [[Rdf example developed during lecture]]


Have a brief a look at this, no need to read thoroughly:
Have a brief a look at this, no need to read thoroughly:
335. rida: 321. rida:
* [https://www.w3.org/RDF/ various rdf tools] listed at the bottom of the page
* [https://www.w3.org/RDF/ various rdf tools] listed at the bottom of the page


=== Lecture: rules in logic with provers to answer questions: Feb 21 ===
What the reasoner does:
 
<!--
[https://livettu.sharepoint.com/:v:/r/sites/Knowledgerepresentation2022/Shared%20Documents/General/Recordings/Meeting%20in%20_General_-20220222_100259-Meeting%20Recording.mp4?csf=1&web=1&e=UzkVBH Loengusalvestus teamsis].
-->
<!--
Recall and go over OWL ideas:
* Official w3c [https://www.w3.org/TR/2012/REC-owl2-primer-20121211/ owl primer] but it is better to look at this [https://www.semantic-web-book.org/w/images/b/b0/KI09-OWL-Rules-1.pdf intro presentation] and continue with [https://www.semantic-web-book.org/w/images/5/5e/KI09-OWL-Rules-2.pdf the owl rules presentation]. If you want, have a deeper look at the [https://www.semantic-web-book.org/ full book].
-->
[https://livettu.sharepoint.com/:v:/s/Knowledgerepresentation2023/Edac0C29IUlPrhnmVmD-mAcBY_Cj2INfwTSczOD5WlMGtA?e=ETbUGu Loengusalvestus teamsis].


We will consider the reasoning part of K&R to understand what provers do:
We will consider the reasoning part of K&R to understand what provers do:
353. rida: 330. rida:
* [[Travel examples from lecture 2021]]
* [[Travel examples from lecture 2021]]


You may want to have a look at the additional materials - from easier to harder - from the [[Media:Loogikaopik.pdf|book chapter pages 297-345]] by Tanel Tammet, the [http://lambda.ee/w/images/0/06/Geoffreasoningnotes.pdf the set of course notes] from Geoff Sutcliffe, [https://resources.mpi-inf.mpg.de/departments/rg1/conferences/vtsa09/slides/schulz.pdf the implementation presentation] from Stephan Schulz and the readable-and-hackable [https://github.com/eprover/PyRes PyRes reasoner] for educational use.
You may want to have a look at the additional materials - from easier to harder - from the  
* [[Media:Loogikaopik.pdf|book chapter pages 297-345]] and optional extras, 346-382; by Tanel Tammet.
* [http://lambda.ee/w/images/0/06/Geoffreasoningnotes.pdf the set of course notes] from Geoff Sutcliffe,  
* [https://resources.mpi-inf.mpg.de/departments/rg1/conferences/vtsa09/slides/schulz.pdf the implementation presentation] from Stephan Schulz  
* [https://fmv.jku.at/rerise14/rerise14-fo-slides.pdf 5-lecture prover course] from the authors of [https://vprover.github.io/ Vampire]
* and the readable-and-hackable [https://github.com/eprover/PyRes PyRes reasoner] for educational use.
 
=== Lecture: DPP ontology development. 2. March ===
 
Loengu annab Riina. Digital Product Passport (DPP)
kui näide praktilisest ontoloogia/sõnastiku arendamisest [https://cirpassproject.eu/dpp-in-a-nutshell/ CIRPASS] projektis.
 
[https://dpp.vocabulary-hub.eu/ Hetkeseisu algusleht].
 
[https://cirpass2.eu/ CIRPASS-2]
 
[https://si-digital-framework.org/SI/units?lang=en Ühikute ontoloogia]
 
[https://www.w3.org/TR/shacl/ SHACL]
 
[https://untp.unece.org/docs/specification/DigitalProductPassport/ UNTP DPP]
 


=== Lecture: what the reasoner does: 28 Feb ===


[https://livettu.sharepoint.com/:v:/s/Knowledgerepresentation2023/Ef3YC_xORFdPnBlLfBc2xnEBY324U3CM1gUQPvMp7E7eew?e=6dHqcc Loengusalvestus teamsis]
<!--
<!--
[https://teams.microsoft.com/l/meetup-join/19%3apbCAi1dcW-0ZYLt9oaEWEF0OVSCh_itNLauu6apjLII1%40thread.tacv2/1677574953174?context=%7b%22Tid%22%3a%223efd4d88-9b88-4fc9-b6c0-c7ca50f1db57%22%2c%22Oid%22%3a%22e3e55dca-e89b-46ae-b8ee-1d158e79281d%22%7d Loengu teamsi link]
 
=== Lecture: what the reasoner does: märtsi algul ===
-->
-->
<!--
<!--
Teamsi [https://livettu.sharepoint.com/:v:/r/sites/Knowledgerepresentation2022/Shared%20Documents/General/Recordings/Meeting%20in%20_General_-20220301_100113-Meeting%20Recording.mp4?csf=1&web=1&e=HrgIfk loengusalvestus 2022 aastast]
<font color="red">NB!</font>This lecture will be the following recording from 2024 pluss self-study reading and experiments only, not in the lecture room:
-->
-->
<!--


We will continue with the same presentation:
[http://turing.cs.ttu.ee/~tanel.tammet/reasoning_engines_recording_2024.mp4 Recording]
* [[Media:Kr_lect_4.pdf| rules, logic, provers]]
along with doing/explaining experiments using [http://logictools.org logictools].
<!--
We will also present the main details of the second practice work.
-->
<!--
* [https://json-ld.org/ json-ld]: currently most popular rdf syntax (in json), also recommended by Google.
-->
-->


== Block 3: General-knowledge databases ==
=== Lecture: Next homework and looking into main large knowledge bases: 9. March  ===
 
We will first present the next homework.
 
Then we will continue with existing large knowledge bases.


=== Lecture: looking into main large knowledge bases: 7. March  ===
Slides:[[Media:ITI8700_05_Large_KB.pdf|lecture slides]] <font color="red">E2</font>, a tool to explore Wordnet taxonomies can be found here: https://github.com/martinve/wntool


[https://livettu.sharepoint.com/:v:/s/Knowledgerepresentation2023/EX-i55YZenZLtLRwgjpOeVIB0IemlS87kmKx94Mxhbm62w?e=SMU7Jf Loengusalvestus 2023]
You can search/investigate what kind of data is available and you can find out actual data from these databases with some work, by surfing on the web pages of the systems.
<!--
[https://livettu.sharepoint.com/:v:/r/sites/Knowledgerepresentation2022/Shared%20Documents/General/Recordings/Teadmiste%20loeng%208%20m%C3%A4rts-20220308_100112-Meeting%20Recording.mp4?csf=1&web=1&e=DyngLd Loengusalvestus teamsis].
-->


<font color="red">E2</font>: you can search/investigate what kind of data is available and you can find out actual data from these databases with some work, by surfing on the web pages of the systems.
About knowlege base creation using LLMs: have a look at [https://arxiv.org/html/2411.04920v1 this paper].


We will have a look at the goals, main content and differences between:
We will have a look at the goals, main content and differences between:
395. rida: 387. rida:
* [http://conceptnet5.media.mit.edu/ conceptnet] described in [https://en.wikipedia.org/wiki/Open_Mind_Common_Sense wikipedia]
* [http://conceptnet5.media.mit.edu/ conceptnet] described in [https://en.wikipedia.org/wiki/Open_Mind_Common_Sense wikipedia]
* [http://rtw.ml.cmu.edu/ nell website] (sometimes down) and in [https://en.wikipedia.org/wiki/Never-Ending_Language_Learning wiki] and [https://cacm.acm.org/magazines/2018/5/227193-never-ending-learning/fulltext a good paper].   
* [http://rtw.ml.cmu.edu/ nell website] (sometimes down) and in [https://en.wikipedia.org/wiki/Never-Ending_Language_Learning wiki] and [https://cacm.acm.org/magazines/2018/5/227193-never-ending-learning/fulltext a good paper].   
* [https://framenet.icsi.berkeley.edu/fndrupal/ framenet] described in [https://en.wikipedia.org/wiki/FrameNet wikipedia], see [https://framenet2.icsi.berkeley.edu/fnReports/data/lu/lu9791.xml example]
* [https://framenet.icsi.berkeley.edu framenet] described in [https://en.wikipedia.org/wiki/FrameNet wikipedia], see [https://framenet2.icsi.berkeley.edu/fnReports/data/lu/lu9791.xml example]
* [https://cyc.com cyc] but just read [https://en.wikipedia.org/wiki/Cyc wikipedia] and see [https://www.cyc.com/wp-content/uploads/2019/09/Cyc-Technology-Overview.pdf whitepaper] also [http://tptp.cs.miami.edu/cgi-bin/SeeTPTP?Category=Axioms&File=CSR002+1.ax tptp small random selection of axioms]
* [https://cyc.com cyc] but just read [https://en.wikipedia.org/wiki/Cyc wikipedia] and see [https://www.cyc.com/wp-content/uploads/2019/09/Cyc-Technology-Overview.pdf whitepaper] also [http://tptp.cs.miami.edu/cgi-bin/SeeTPTP?Category=Axioms&File=CSR002+1.ax tptp small random selection of axioms]
* [https://adimen.si.ehu.es/web/adimenSUMO adimen-sumo] see [https://en.wikipedia.org/wiki/Suggested_Upper_Merged_Ontology sumo in wikipedia] and [http://tptp.cs.miami.edu/cgi-bin/SeeTPTP?Category=Axioms&File=CSR003+0.ax tptp axioms]
* [https://www.ontologyportal.org/ sumo] and [https://adimen.ehu.eus/web/adimenSUMO adimen-sumo], see [https://en.wikipedia.org/wiki/Suggested_Upper_Merged_Ontology sumo in wikipedia] and [http://tptp.cs.miami.edu/cgi-bin/SeeTPTP?Category=Axioms&File=CSR003+0.ax tptp axioms]
* [http://tptp.cs.miami.edu/ tptp]: a large set of axioms and problems in logic usable for automated reasoners
* [https://www.tptp.org/ tptp]: a large set of axioms and problems in logic usable for automated reasoners
* [http://schema.org schema.org]: property markup vocabulary suggested by Google, Microsoft and others.
* [http://schema.org schema.org]: property markup vocabulary suggested by Google, Microsoft and others.
* [https://reference.wolfram.com/language/guide/KnowledgeRepresentationAndAccess.html Wolfram alpha stuff]
* [https://reference.wolfram.com/language/guide/KnowledgeRepresentationAndAccess.html Wolfram alpha stuff]
428. rida: 420. rida:
-->
-->


== Block 4: natural language and question answering ==
== Block 3: natural language and question answering ==


=== Lecture: Intro to NLP, n-grams, word vectors: 14 March ===
=== Lecture: Intro to NLP, n-grams, word vectors: 16 March ===


<!--
[https://livettu.sharepoint.com/:v:/r/sites/Knowledgerepresentation2024/Shared%20Documents/General/Recordings/Teadmiste%20loeng%205%20m%C3%A4rts-20240305_120418-Meeting%20Recording.mp4?csf=1&web=1&e=cWEU8A&nav=eyJyZWZlcnJhbEluZm8iOnsicmVmZXJyYWxBcHAiOiJTdHJlYW1XZWJBcHAiLCJyZWZlcnJhbFZpZXciOiJTaGFyZURpYWxvZy1MaW5rIiwicmVmZXJyYWxBcHBQbGF0Zm9ybSI6IldlYiIsInJlZmVycmFsTW9kZSI6InZpZXcifX0%3D Loengusalvestus teamsis]
-->
<!--
[https://livettu.sharepoint.com/:v:/s/Knowledgerepresentation2023/Ea7HdMQxBcBGlSqBi18ZC2cBg7HU371GuQOVkdvkDjnyVg?e=PNaowE Loengusalvestus]
[https://livettu.sharepoint.com/:v:/s/Knowledgerepresentation2023/Ea7HdMQxBcBGlSqBi18ZC2cBg7HU371GuQOVkdvkDjnyVg?e=PNaowE Loengusalvestus]
-->


* These three wikipages give useful introductory details:[https://en.wikipedia.org/wiki/Natural_language_processing Natural language processing], [https://en.wikipedia.org/wiki/Knowledge_extraction Knowledge extraction], [https://en.wikipedia.org/wiki/Natural_language_understanding Natural language understanding].
* These three wikipages give useful introductory details:[https://en.wikipedia.org/wiki/Natural_language_processing Natural language processing], [https://en.wikipedia.org/wiki/Knowledge_extraction Knowledge extraction], [https://en.wikipedia.org/wiki/Natural_language_understanding Natural language understanding].
* Väga hea õpik: [https://web.stanford.edu/~jurafsky/slp3/ 3. variandi draft] või [https://github.com/rain1024/slp2-pdf/blob/master/complete-book-pdf/slp2.pdf 2. variandi terve pdf].
* Tanel Alumäe keelemudelitesse [http://lambda.ee/w/images/1/13/Keelemudelid.pdf sissejuhatavad slaidid] <font color="red">E1</font> ja [https://echo360.org.uk/media/e4323062-3442-46fb-9124-e7b9e7d00006/public loengusalvestus]
** Hea mõte on hakatuseks lugeda läbi [https://github.com/rain1024/slp2-pdf/blob/master/complete-book-pdf/slp2.pdf 2. variandi] sissejuhatus.
* Väga hea õpik: [https://web.stanford.edu/~jurafsky/slp3/ 3. variant].  
** Vektorsemantikast arusaamiseks loe 3. variandi [https://web.stanford.edu/~jurafsky/slp3/3.pdf ptk 3. N-gram Language Models] ja siis [https://web.stanford.edu/~jurafsky/slp3/6.pdf chapter 6: Vector Semantics and Embeddings] <font color="red">E2</font>. Vektorsemantika kasutamine närvivõrkudes a la Bert ja GPT on hästi kirjas ptks 7: Neural Networks and Neural Language Models
** Vektorsemantikast arusaamiseks loe 3. variandi [https://web.stanford.edu/~jurafsky/slp3/3.pdf ptk 3. N-gram Language Models] <font color="red">E2</font> ja siis [https://web.stanford.edu/~jurafsky/slp3/5.pdf chapter 5: Embeddings] <font color="red">E2</font>. Vaata [https://web.stanford.edu/~jurafsky/slp3/ sisukorrast] ka slaide nende kohta.
** Slaidid eelmiste peatükkide kohta: [https://web.stanford.edu/~jurafsky/slp3/slides/3_LM_Jan_08_2021.pdf Ngrams] ja [https://web.stanford.edu/~jurafsky/slp3/slides/6_Vector_Apr18_2021.pdf word vectors].
* Google [https://books.google.com/ngrams/info ngrams] ja [https://storage.googleapis.com/books/ngrams/books/datasetsv3.html reaalne data].
* Google [https://books.google.com/ngrams/info ngrams] ja [https://storage.googleapis.com/books/ngrams/books/datasetsv3.html reaalne data].
* [https://nlp.stanford.edu/projects/glove/ GloVE]: a relatively simple vector representation
* [https://nlp.stanford.edu/projects/glove/ GloVE]: a relatively simple vector representation
* [https://openai.com/blog/chatgpt ChatGPT]: katseta!
<!--
<!--
* [https://beta.openai.com/ GPT-3 API beta] publicly available without a waiting period. There is also a playground on the web.  
* [https://beta.openai.com/ GPT-3 API beta] publicly available without a waiting period. There is also a playground on the web.  
-->
-->
* Kui tahad katsetada toore Wikipedia dataga, siis saab loengus vaadatud töödeldud variante siit (tarballid sisaldavad datat, selgitavat README-d ja tarkvara selle ise ehitamiseks): [http://turing.cs.ttu.ee/~tanel.tammet/wikicompact.tar.gz  a compacted pure-text version of full wikipedia], [http://turing.cs.ttu.ee/~tanel.tammet/wikilemma.tar.gz A lemmatized version of wikipedia texts], [http://turing.cs.ttu.ee/~tanel.tammet/wikicooccurrence.tar.gz Several co-occurrence matrices and lists of top-co-occurring words for wikipedia]
* Kui tahad katsetada toore Wikipedia dataga, siis saab loengus vaadatud töödeldud variante siit (tarballid sisaldavad datat, selgitavat README-d ja tarkvara selle ise ehitamiseks): [http://turing.cs.ttu.ee/~tanel.tammet/wikicompact.tar.gz  a compacted pure-text version of full wikipedia], [http://turing.cs.ttu.ee/~tanel.tammet/wikilemma.tar.gz A lemmatized version of wikipedia texts], [http://turing.cs.ttu.ee/~tanel.tammet/wikicooccurrence.tar.gz Several co-occurrence matrices and lists of top-co-occurring words for wikipedia]
* Online [http://text-processing.com/demo/ demo] of NTLK running some basic NLP tasks
<!--
<!--
A lot of reading this week. The main material is in the following chapters 23 and 24 of the Jurafsky and Manning book, the rest is relevant overviews/background. Lecture materials for the week:
A lot of reading this week. The main material is in the following chapters 23 and 24 of the Jurafsky and Manning book, the rest is relevant overviews/background. Lecture materials for the week:
454. rida: 448. rida:
* Read from the Jurafsky and Martin book: [https://web.stanford.edu/~jurafsky/slp3/24.pdf chapter 24: Dialogue Systems and Chatbots] <font color="red">E2</font>and the supporting slides [https://web.stanford.edu/~jurafsky/slp3/slides/convagents1.pdf for conversational agents]
* Read from the Jurafsky and Martin book: [https://web.stanford.edu/~jurafsky/slp3/24.pdf chapter 24: Dialogue Systems and Chatbots] <font color="red">E2</font>and the supporting slides [https://web.stanford.edu/~jurafsky/slp3/slides/convagents1.pdf for conversational agents]
-->
-->
 
<!--
Recommended listening:
Recommended listening:


* A cool 14-minute philosophy podcast episode to listen: [https://philosophybites.com/2015/03/jonathan-webber-on-deceiving-with-words.html Jonathan Webber on deceiving with words] plus a [https://hwcdn.libsyn.com/p/c/f/7/cf74e80b83d07419/Jonathan_Webber_on_Deception_With_Words.mp3?c_id=8457944&cs_id=8457944&expiration=1584653784&hwt=286ccb5004366328b50492800a250f57 a direct audio link (may expire at some point)]
* A cool 14-minute philosophy podcast episode to listen: [https://nigelwarburton.typepad.com/philosophy_bites//2015/03/jonathan-webber-on-deceiving-with-words.html Jonathan Webber on Deception With Words] plus a [http://traffic.libsyn.com/philosophybites/Jonathan_Webber_on_Deception_With_Words.mp3 a direct audio link (may expire at some point)]
* [https://soundcloud.com/nlp-highlights NLP highlights] podcast series from the [https://allenai.org/ Allen institute for AI]. Makes sense to start from the very first ones and pick these which have a title looking interesting to you. World-leading stuff and quite understandable.
* A bit old (i.e. few new episodes) [https://soundcloud.com/nlp-highlights NLP highlights] podcast series from the [https://allenai.org/ Allen institute for AI]. Makes sense to start from the very first ones and pick these which have a title looking interesting to you. World-leading stuff and quite understandable.
-->
* Loengus tehtud [[katse Glove vektoritega]] [https://medium.com/analytics-vidhya/basics-of-using-pre-trained-glove-vectors-in-python-d38905f356db selle jutu] järgi.


=== Lecture: large language models (LLM), BERT and GPT families: 21 märts ===
=== Lecture: large language models (LLM) 23 March ===


Iseõppe nädal, loengut ei toimu (õppejõud peab olema sama ajal senati istungil).
* [[Media:Llm_intro_2026.pdf|Lecture material part 1]] <font color="red">E2</font>
* [[Media:Llm_details_2026.pdf|Lecture material part 2]] <font color="red">E2</font>
 
<!--
[https://livettu.sharepoint.com/:v:/r/sites/Knowledgerepresentation2024/Shared%20Documents/General/Recordings/New%20channel%20meeting-20240312_120339-Meeting%20Recording.mp4?csf=1&web=1&e=j8fZj2&nav=eyJyZWZlcnJhbEluZm8iOnsicmVmZXJyYWxBcHAiOiJTdHJlYW1XZWJBcHAiLCJyZWZlcnJhbFZpZXciOiJTaGFyZURpYWxvZy1MaW5rIiwicmVmZXJyYWxBcHBQbGF0Zm9ybSI6IldlYiIsInJlZmVycmFsTW9kZSI6InZpZXcifX0%3D loengusalvestus koos teise praksi ülesannetega]
-->


Eesmärk: saada <i>pinnaliselt</i> aru LLM ja transformerite (NLP masinõppe põhiasjad) peamistest ideedest. Detailid on tegelikult keerulised ja nendest arusaamine võtab hulga rohkem aega, kui meil kursuses on, sestap lepime vähesega. Samas, kui sul endal huvi, siis loomulikult on teretulnud detailsem iseõpe.
Eesmärk: saada <i>pinnaliselt</i> aru LLM ja transformerite (NLP masinõppe põhiasjad) peamistest ideedest. Detailid on tegelikult keerulised ja nendest arusaamine võtab hulga rohkem aega, kui meil kursuses on, sestap lepime vähesega. Samas, kui sul endal huvi, siis loomulikult on teretulnud detailsem iseõpe.


Enne järgmisse materjali süvenemist tutvu kindlasti vektorsemantikaga, mida seletasime 14 märtsi loengus.
Enne järgmisse materjali süvenemist tutvu kindlasti vektorsemantikaga, mida seletasime 11 märtsi loengus.


Mida vaadata ja lugeda:
Mida vaadata ja lugeda:


Kindlasti
Eriti soovitav


* [https://www.youtube.com/watch?v=kCc8FmEb1nY Andrej Karpathy progemisega-koos loeng]: vaata see süvenemisega läbi!
* [https://www.youtube.com/watch?v=kCc8FmEb1nY Andrej Karpathy progemisega-koos loeng]: vaata see süvenemisega läbi! Vt ka [https://en.wikipedia.org/wiki/Andrej_Karpathy Karpathy].


Paratamatult jääb enamus detaile loengus vähearusaadavaks (kui ise ei kuluta hulga aega järele katsetamiseks), aga sellegipoolest on see tõenäoliselt kõige parem loeng transformeritest: mingi tunnetuse sellest, et mis toimub, saab.
Paratamatult jääb enamus detaile loengus vähearusaadavaks (kui ise ei kuluta hulga aega järele katsetamiseks), aga sellegipoolest on see tõenäoliselt kõige parem loeng transformeritest: mingi tunnetuse sellest, et mis toimub, saab.
Pikemaks lugemiseks:
* [https://jalammar.github.io/visualizing-neural-machine-translation-mechanics-of-seq2seq-models-with-attention/ visualizing neural machine translation] and then [https://jalammar.github.io/illustrated-transformer/ the illustrated transformer]
* [https://web.stanford.edu/~jurafsky/slp3/7.pdf Jurafsky & Martin book: neural networks] and the [https://web.stanford.edu/~jurafsky/slp3/slides/7_NN_Apr_28_2021.pdf slides]. Alternative: [http://www.cs.cmu.edu/afs/cs.cmu.edu/project/theo-20/www/mlbook/ch4.pdf CMU Mitchell course: neural networks basics]
* [https://web.stanford.edu/~jurafsky/slp3/9.pdf Jurafsky & Martin: chapter 9, transformers ]
<!-- * [https://web.stanford.edu/~jurafsky/slp3/10.pdf Jurafsky & Martin: chapter 10, transformers and LLMs] -->
<!-- * [https://dugas.ch/artificial_curiosity/GPT_architecture.html GPT arhitektuuri üldseletus]: eriti head intuitsiooni ei maksa loota :) -->


Optsionaalselt üldisem arusaam klassikutelt:
Optsionaalselt üldisem arusaam klassikutelt:


* [https://lexfridman.com/yann-lecun-3 Jann LeCun: värske episood Lex Fridmani podcastist]
* [https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ Wolframi pikk jutt] lugemiseks koos närvivõrkudega üldisema taustaga
* [https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ Wolframi pikk jutt] lugemiseks koos närvivõrkudega üldisema taustaga
* [https://rodneybrooks.com/what-will-transformers-transform/ Rodney Brooksi] ennustused transformerite kohta
* [https://rodneybrooks.com/what-will-transformers-transform/ Rodney Brooksi] ennustused transformerite kohta
483. rida: 493. rida:
Optsionaalselt diipim arusaam:  
Optsionaalselt diipim arusaam:  


* [https://github.com/karpathy/nanoGPT nanoGPT]: Karpathy reaalne näitekood GPT ise-ehitamiseks (otse eelmise loenguga seotud) : seda saab ilma suuremate pingutusteta käima linuxil ja macil, windowsil tekib aga suuremaid probleeme (proovisin ise linuxil järgi).  
* [https://github.com/karpathy/nanoGPT nanoGPT]: [Karpathy reaalne näitekood GPT ise-ehitamiseks (otse eelmise loenguga seotud) : seda saab ilma suuremate pingutusteta käima linuxil ja macil, windowsil tekib aga suuremaid probleeme (proovisin ise linuxil järgi).  
* [https://karpathy.ai/zero-to-hero.html Taustaks: Karpathy terve loenguseeria], kui sul tekib sügavam huvi
* [https://karpathy.ai/zero-to-hero.html Taustaks: Karpathy terve loenguseeria], kui sul tekib sügavam huvi
* [https://web.stanford.edu/~jurafsky/slp3/10.pdf Jurafsky & Martin ptk 10 Transformers and Pretrained Language Models] kui tahad ise juurde lugeda.
* [https://web.stanford.edu/~jurafsky/slp3/10.pdf Jurafsky & Martin ptk 10 Large Language Models] kui tahad ise juurde lugeda.
* Good free (?) book:  [https://www.manning.com/books/build-a-large-language-model-from-scratch Build a Large Language Model (From Scratch)]
* [https://course.fast.ai/ väga detailne ja mahukas videoloengutega deep learningu kursus], mh [https://course.fast.ai/Resources/book.html raamat].


<!--
=== Lecture: RAG usage pattern / context engineering for LLMs : 30 March ===
 
[[Media:Llm_usage_rag.pdf|Lecture material]]: this is focused on RAG and written in 2024 and spring 2025: good tool use, good reasoning and more autonomy like claude code and openclaw are not covered in this overview.
 
<b>"Context Engineering"</b> became a popular phrase in 2025: essentially it means "fancy RAG", i.e. significant effort put towards creating a really good set of additional information (prompt additions: "context") for each user query/prompt. It can also mean approaches for storing such additional information on the fly in special files which
are then used RAG-like as extra input in addition to user dialog.For example, claude code [https://code.claude.com/docs/en/memory memory principles] and [https://blog.dailydoseofds.com/p/anatomy-of-the-claude-folder memory folder structure], openclaw [https://lumadock.com/tutorials/openclaw-memory-explained memory].


[https://livettu.sharepoint.com/:v:/r/sites/Knowledgerepresentation2022/Shared%20Documents/General/Recordings/Teadmiste%20kursuse%20loeng%2029%20m%C3%A4rts-20220329_100211-Meeting%20Recording.mp4?csf=1&web=1&e=FV87X7 Loengusalvestus] teamsis.
Useful reading in addition to pointers in the lecture material:
* [https://pixelplex.io/blog/llm-applications/ application landscape overview]
<!-- * early taxonomy [https://arxiv.org/pdf/2311.05288.pdf of real business use] -->
* RAG focus: [https://eugeneyan.com/writing/llm-patterns/ LLM improvement for applications] and a basic overview of [https://www.databricks.com/glossary/retrieval-augmented-generation-rag customizing llms]
* Old finetuning example: [https://www.anyscale.com/blog/fine-tuning-llama-2-a-comprehensive-case-study-for-tailoring-models-to-unique-applications Finetuning llama 2]
* RAG:
** [https://colabdoge.medium.com/what-is-rag-retrieval-augmented-generation-b0afc5dd5e79 simplest RAG intro] <font color="red">E1</font>
** [https://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00530/114590/Improving-the-Domain-Adaptation-of-Retrieval more rag details]: a good architecture picture, may skip the improvements part after that
** [https://medium.com/@evertongomede/understanding-sequence-to-sequence-seq2seq-models-and-their-significance-d2f0fd5f6f7f encoder/decoder] and [https://en.wikipedia.org/wiki/Seq2seq Seq2seq] as a generalization of [https://en.wikipedia.org/wiki/Word2vec word2vec] and [https://openai.com/blog/new-and-improved-embedding-model openai embedding model api]
** [https://en.wikipedia.org/wiki/Vector_database vector database] <font color="red">E2</font> as a useful tool
** [https://www.promptingguide.ai/research/rag different kinds of rag] <font color="red">E2</font> based on [https://arxiv.org/abs/2312.10997 paper]
<!--
** [https://python.langchain.com/docs/expression_language/cookbook/retrieval rag by langchain]
** [https://proceedings.neurips.cc/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html a fancy rag paper]
-->
<!-- ** [https://getalign.com/ example estonian startup using a kind of rag] -->


Nädala eesmärk on konkreetne: <b>sul on vaja aru saada vektorsemantikast</b>, ehk, mis see on, kuidas sõnavektoreid moodustatakse ja kuidas neid kasutada saab.
== Block 4: hybrid systems and uncertain knowledge ==


Now, you may have had a course on NLP and machine learning, in which case you should already know <b>a lot</b> about vector semantics. If so, consider yourself lucky :) and just skim through the following materials to check whether there are any new things or old friends for you. If not so, consider yourself extra lucky: you have come to the core of machine-learning-based NLP, and this is both pretty cool and quite understandable at the same time! Most of the machine-learning-based NLP is built on top of the ideas of vector representation of words.
We will focus on (a) world modeling and uncertainty in KR (open/closed world, frame problem, probabilities, exceptions) and (b) current research on building hybrid systems


As a continuation of the lecture from the last week, please have a look and try out
=== Lecture: issues in formalizing commonsense reasoning: open/closed world, frame problem, uncertainties: 6. April ===
* [https://demo.allennlp.org/ Allen NLP demos] for a number of core NLP techniques: highly recommended for experimentation.


Background study materials for vector representations (continuation of the previous lecture):
<!--
* First, listen to [http://lineardigressions.com/episodes/2017/4/23/feature-processing-for-text-analytics text vectorization episode] from the [http://lineardigressions.com/ linear digressions] podcast.
<i>First, some results/challenges in neurosymbolic reasoning.</i>
* Second, listen to [http://lineardigressions.com/episodes/2018/12/30/re-release-word2vec word2vec episode] from the [http://lineardigressions.com/ linear digressions] podcast.
* Finally the <b>main thing</b>: slowly read the whole [https://web.stanford.edu/~jurafsky/slp3/6.pdf Vector Semantics: chapter 6] <font color="red">E2</font> from the Jurafsky & Martin book.


Additional supporting non-obligatory materials from (roughly) easier to more complex:
* [https://arxiv.org/pdf/2301.13379.pdf generating computation tasks]
* [https://github.com/tammet/llmlog our current experiments in reasoning with LLM]
* [https://onlinelibrary.wiley.com/doi/full/10.1111/cogs.13432 neurosymbolic Bongard]
* [https://arcprize.org/ arcprize]. See also [https://liusida.github.io/ARC/ ARC v1 for humans], [https://www.youtube.com/watch?v=jSAT_RuJ_Cg MLST about the state of v1 solvers], [https://www.youtube.com/watch?v=M3b59lZYBW8 MLST video introducing ARC v2]
* [https://deepmind.google/discover/blog/alphageometry-an-olympiad-level-ai-system-for-geometry/ alphageometry]
* [https://arxiv.org/pdf/2402.01817.pdf a position paper on hybrid systems] <font color="red">E2</font>
-->
-->
<!-- * [https://medium.com/@jayeshbahire/introduction-to-word-vectors-ea1d4e4b84bf word vectors]: a short intro -->
<!--
<!--
* [https://web.stanford.edu/~jurafsky/slp3/slides/vector1.pptx vector semantics part I] presentation from the Jurafsky & Martin book for for chapter 6.
* [https://arxiv.org/pdf/2302.07842.pdf a large survey of tool use]  
* [https://towardsdatascience.com/art-of-vector-representation-of-words-5e85c59fee5 vector representation]: a short tutorial with code examples.
* [http://u.cs.biu.ac.il/~gonenhi/slides/WiDS_tutorial.pdf vector representation and word2vec]: a medium-sized tutorial presentation.
* [https://www.tensorflow.org/tutorials/representation/word2vec word2vec tutorial] from [https://www.tensorflow.org/ tensorflow]
* [https://web.stanford.edu/~jurafsky/slp3/slides/vector2.pptx vector semantics part II] <font color="red">E2</font> presentation from the Jurafsky & Martin book for chapter 6.
-->
-->
<!--* [[Media:How_does_word2vec_work.pdf|How does word2vec work]] paper by David Meyer.
* [[Media:Word2vec_learning_explained.pdf|Word2vec learning explained]] paper by Xin Rong.-->
<!--
<!--
If you want to get a deeper understanding and look at longer lectures while doing that, there is a wealth of good word vector lecture videos available,
See also an [https://adamkarvonen.github.io/machine_learning/2024/01/03/chess-world-models.html interesting recent piece about learning chess]
for example, from the Stanford NLP course
* [https://www.youtube.com/watch?v=IYMYI9AJpQs Word vectors 1]
* [https://www.youtube.com/watch?v=nH4rn3X8i0c Word vectors 2]
* [https://www.youtube.com/watch?v=pip8h9vjTHY Word vectors 3]
However, I'd recommend to use the obligatory and supporting materials above first and only - optionally - spend time with the longer lectures after that.


Also interesting to look and read:
We will first give a brief overview / reminder of the work/tasks in hybrid neurosymbolic AI.
* Probabilistic models: [https://www.youtube.com/watch?v=u7l5hhmdc0M LDA video explanation by Jordan Boyd-Graber]
* [https://drive.google.com/file/d/0B8i61jl8OE3XdHRCSkV1VFNqTWc/edit Aetherial Symbols] by the ultimate machine learning guru aka "godfather of machine learning" [https://en.wikipedia.org/wiki/Geoffrey_Hinton Geoffrey Hinton]
*[https://yurichev.com/blog/markov/ Autocomplete using Markov chains]
-->
<!--
=== Lecture: large language models, BERT and GPT families ===


Materials will be added later.
We will then consider the issues regarding the world model and the output we expect from
the neural part / LLM, along with some demonstrations of semantic parsing with and without the use of LLM.
-->
-->


== Block 5: uncertain knowledge and hybrid systems ==
We will focus on (a) uncertainty in KR (open/closed world, frame problem, probabilities, exceptions) and (b) current research on building hybrid systems
=== Lecture: third lab, neurosymbolic reasoning, open/closed world, planning, frame problem, blocks world: 28 March ===
We will first have a brief look at the 3. lab: [[Experiment with natural language rules, ChatGPT and ProofWriter]] and an alternative [[Use a semantic parser and a reasoner to solve questions posed in natural language]].
<!--
[https://livettu.sharepoint.com/:v:/r/sites/Knowledgerepresentation2022/Shared%20Documents/General/Recordings/General-20220405_100120-Meeting%20Recording.mp4?csf=1&web=1&e=IbnqJe 202 loengusalvestus teamsis]
-->


We will then give a brief overview of the work/tasks in hybrid neurosymbolic AI.
<b><i>First, open/closed world, world states, frame problem</i></b>


For the main part of the lecture we consider planning and the frame problem. The standard example for these
The standard example for these is a blocks world: some blocks are on the table and the robot arm can lift single blocks
is a blocks world: some blocks are on the table and the robot arm can lift single blocks
and put them on other blocks or on the table. The arm cannot lift several blocks or a tower
and put them on other blocks or on the table. The arm cannot lift several blocks or a tower
of blocks. Then, given a position of blocks at initial situation, can the robot arm create a
of blocks. Then, given a position of blocks at initial situation, can the robot arm create a
559. rida: 565. rida:


Importantly, the frame problem arises since normal first order logic has  
Importantly, the frame problem arises since normal first order logic has  
* the [https://en.wikipedia.org/wiki/Open-world_assumption open world assumption] <font color="red">E1</font> (must read): if we do not have (and cannot derive) a positive fact like P(a) and neither a negative fact like -P(a), we assume we simply do not know which holds.
* the <i>open world assumption</i>: if we do not have (and cannot derive) a positive fact like P(a) and neither a negative fact like -P(a), we assume we simply do not know which holds.


Prolog and databases operate under the  
Prolog and databases operate under the  
566. rida: 572. rida:


Please look at and read these materials about the frame problem:
Please look at and read these materials about the frame problem:
 
<!--
* [https://www.youtube.com/watch?v=JzlsvFN_5HI tiny video] of the Boston Dynamics robot failing: looks like a frame problem in the blocks world :)
* [https://www.youtube.com/watch?v=JzlsvFN_5HI tiny video] of the Boston Dynamics robot failing: looks like a frame problem in the blocks world :)
-->
* [https://people.cs.pitt.edu/~milos/courses/cs2740/Lectures/class14.pdf overview presentation]: <font color="red">E1</font> must read.
* [https://people.cs.pitt.edu/~milos/courses/cs2740/Lectures/class14.pdf overview presentation]: <font color="red">E1</font> must read.
* [https://folk.idi.ntnu.no/gamback/teaching/TDT4138/dennett84.pdf a classic AI philosophy paper] about the frame problem: must read the first three pages (rest is optional).
* [https://folk.idi.ntnu.no/gamback/teaching/TDT4138/dennett84.pdf a classic AI philosophy paper] about the frame problem: must read the first three pages (rest is optional).
580. rida: 587. rida:
Next, importantly, you should experiment yourself with gkc and both of the following
Next, importantly, you should experiment yourself with gkc and both of the following
files. Please copy and read the files and understand the encoding. At the end of the files are
files. Please copy and read the files and understand the encoding. At the end of the files are
several queries to experiment with: instructions are also there.
several queries to experiment with: instructions are also there. The simplest way to experiment is to use https://logictools.org/ . There is already a <b>blocks world</b> example available.


In both of them the predicate holds(X,Y) is used to describe something X about the blocks world (like, one block is on another or robot holds some block) and the second argument Y describes the state we have arrived to via robot actions from the original state (like, picked up a block in the initial state and then did put the block down on top of another).
In both of them the predicate holds(X,Y) is used to describe something X about the blocks world (like, one block is on another or robot holds some block) and the second argument Y describes the state we have arrived to via robot actions from the original state (like, picked up a block in the initial state and then did put the block down on top of another).
586. rida: 593. rida:
* [[Simple axiomatization of the blocks world]]: <font color="red">E2</font> please also read through the comments explaining the axiomatization and queries!
* [[Simple axiomatization of the blocks world]]: <font color="red">E2</font> please also read through the comments explaining the axiomatization and queries!
* [[More complex blocks world axiomatization]] taken from the [http://www.tptp.org/ TPTP] problem set for first order logic, concretely by concatenanting and commenting axiom sets [[PLA001-0.ax]], [[PLA001-1.ax]] and queries [[PLA004-1.p]], [[PLA005-1.p]], [[PLA019-1.p]]
* [[More complex blocks world axiomatization]] taken from the [http://www.tptp.org/ TPTP] problem set for first order logic, concretely by concatenanting and commenting axiom sets [[PLA001-0.ax]], [[PLA001-1.ax]] and queries [[PLA004-1.p]], [[PLA005-1.p]], [[PLA019-1.p]]


To get a simple visualization of the blocks world, have a look at this [https://www.youtube.com/watch?v=gbIPicRoIgQ tiny video].
To get a simple visualization of the blocks world, have a look at this [https://www.youtube.com/watch?v=gbIPicRoIgQ tiny video].


<!--
<b>NB!</b>Please email me tanel.tammet at taltech.ee a short description of (a) with which of the queries in the files were you successful and how long did they approximately take, (b) what queries did you invent yourself and
what happened with these. Please put a word "formaliseerimine" into the email subject so I can easily find them.
-->


=== Reasoning with uncertainty: 4 April ===
<b><i>Second, uncertainty</i></b>


* [https://livettu.sharepoint.com/:v:/s/Knowledgerepresentation2022/EUZLJOrQonJNuQG5uMr6TSwBxV6b91HJcPz1XRG9gRgHlA?e=J1KIbS Loengusalvestus 2022: siit vaata esimest 30 minutit].
* [https://livettu.sharepoint.com/:v:/s/Knowledgerepresentation2023/EQXdkEsVEeFEp4BJBIQIp44BUhezJHLiOe4Ag6NAGaFX3g?e=PedhX6 Loengusalvestus 2023: uuem variant pärast eelmise 30 minutit ja edasi]


Loengu sissejuhatava osa presentatsioon: [[Media:Reasoning_with_uncertainty.pdf|Reasoning_with_uncertainty.pdf]] <font color="red">E1</font>.
Loengu sissejuhatava osa presentatsioon: [[Media:Reasoning_with_uncertainty.pdf|Reasoning_with_uncertainty.pdf]] <font color="red">E1</font>.
612. rida: 610. rida:
NB! Both of these ways are hard to actually implement, neither have they been understood very well, despite an immense amount of research and papers. For example, typical machine learning methods operate with extremely vague probability-like numbers and do not attempt to put these numbers into a real properly-theoretical-and-correct probabilities framework.
NB! Both of these ways are hard to actually implement, neither have they been understood very well, despite an immense amount of research and papers. For example, typical machine learning methods operate with extremely vague probability-like numbers and do not attempt to put these numbers into a real properly-theoretical-and-correct probabilities framework.


The main material for reading about numeric methods is an overview-followed-in-depth by the lecturer <b>[[Numeric uncertainty]]</b>: (<font color="red">E2</font> up to and including the chapter 3, see also <font color="red">E2</font> tags inside)  
The main material for reading about numeric methods is an overview-followed-in-depth by the lecturer [[Numeric uncertainty]]: (<font color="red">E2</font> up to and including the chapter 3, see also <font color="red">E2</font> tags inside)  
* <b>Read carefully</b> up to and including the chapter 4 "Different ways to encode confidences in logic" and have a quick look at all the wikipedia links.
* <b>Read carefully</b> up to and including the chapter 4 "Different ways to encode confidences in logic" and have a quick look at all the wikipedia links. In particular, [https://en.wikipedia.org/wiki/Bayesian_inference Wiki on Bayesian inference] and [https://math.mit.edu/~dav/05.dir/class11-prep.pdf this]
* Skim through the later parts and look only into these subchapters which you found interesting.
* Skim through the later parts and look only into these subchapters which you found interesting.


Next,  
Next,  
* Read [https://link.springer.com/chapter/10.1007/978-3-030-79876-5_29 the paper] <font color="red">E2</font>. containing intro, alorithms and experiments.
* Read [https://link.springer.com/chapter/10.1007/978-3-030-79876-5_29 the paper] <font color="red">E2</font> about confidences. containing intro, algorithms and experiments. Then look at the [http://logictools.org/confer/ web page] with the confer reasoner and a wealth of examples.
* Look at the [http://logictools.org/confer/ web page] with the confer reasoner and a wealth of examples.
* Read [https://link.springer.com/chapter/10.1007/978-3-031-10769-6_18 the paper] <font color="red">E2</font> about exceptions (default logic). containing intro, alorithms and experiments. Then look at the [http://logictools.org/gk/ web page] with the gk reasoner and a wealth of examples.


Next, have a brief look at a good example system doing some specific kinds of probability reasoning:
Next, have a brief look at a good example system doing some specific kinds of probability reasoning:
* [https://dtai.cs.kuleuven.be/problog/ Problog] start with a [https://dtai.cs.kuleuven.be/problog/tutorial.html tutorial] <font color="red">E2</font>
* [https://dtai.cs.kuleuven.be/problog/ Problog] start with a [https://dtai.cs.kuleuven.be/problog/tutorial.html tutorial] <font color="red">E2</font>
* Try out some examples via browser from the [https://dtai.cs.kuleuven.be/problog/tutorial.html tutorial]
* Try out some examples via browser from the [https://dtai.cs.kuleuven.be/problog/tutorial.html tutorial]
<!--
Finally some additional possibly helpful material considering some parts of the main [[Numeric uncertainty]] material.
* [[Media:Uncertain_prob_fuzzy.ppt?|Uncertain_prob_fuzzy.ppt?]] a good intro to both Bayesian and Fuzzy reasoning.
* [[Media:Vienna_tanel_4.pdf|Vienna_tanel_4.pdf]] some additional examples of different uncertainties and possible combinations of these.
-->


=== Lecture: initial reviews of GPT4 + more on 3rd pract work + semantic parsing, 11 April ===
=== Lecture: research in neurosymbolic reasoning: 13 April ===


[https://livettu.sharepoint.com/:v:/s/Knowledgerepresentation2023/EWMpX54Kw8JAiF1hIMGWe-QBp_fy9DE6eJpjtYBR0RIq9w?e=dgTfKe&nav=eyJwbGF5YmFja09wdGlvbnMiOnsic3RhcnRUaW1lSW5TZWNvbmRzIjoxNTEuNX19 Lecture recording 2023]
[[Media:Research_in_Neurosymbolic_Reasoning.pdf|lecture slides]]


<!--
The lecture will be given by our PhD student [https://aire-edih.eu/meeskond/otto-mattas/ Otto Mättas].
[https://livettu.sharepoint.com/:v:/r/sites/Knowledgerepresentation2022/Shared%20Documents/General/Recordings/Teadmiste%20loeng%2019%20aprill-20220419_100152-Meeting%20Recording.mp4?csf=1&web=1&e=lgh5OR Lecture recording 2022].
-->
Materials accompanying the lecture:
* [https://arxiv.org/abs/2303.12712 Paper "Sparks of Artificial General Intelligence: Early experiments with GPT-4"]
* [https://github.com/tammet/nlpsolver nlpsolver] and [[Media:nlp_reasoning_pipeline.pdf]]
* [[nlpsolver knowledge representation principles: draft]]


Additionally recommended to listen and read in this order:
=== Lecture: semantic parsing with LLMs and with UDP: 20 April ===


<!--
The focus of the lecture is on [https://en.wikipedia.org/wiki/Semantic_parsing semantic parsing].
* Read through the [[convert English questions to logical queries]]
-->
* Listen to the episode [https://lexfridman.com/david-ferrucci/ David Ferrucci: IBM Watson, Jeopardy & Deep Conversations with AI]  in a very popular [https://lexfridman.com/ai/ ai podcast]. <!-- Previous recommendation [https://voicesinai.com/episode/episode-55-a-conversation-with-rob-high/ of the Rob High episode] in the [https://voicesinai.com/episodes/ voices in AI] podcast series is also ok, but not so good (may skip that).-->
* Read a [https://yoavartzi.com/sp14/slides/mooney.sp14.pdf good presentation] <!<!-- <font color="red">E2</font> --> about a history of semantic parsing: this won't tell you much about how to do it, though.
<!-- You may want to listen a [https://omegataupodcast.net/122-artificial-intelligence/ podcast with the author of the presentation]: this is not obligatory, but is kind of easy listening. -->


Also, check out a very useful demo:
For a quick intro we will use Martin's slides from 2024 [[Media:KR_2024_Parsing_01.pdf|semantic parsing]] <font color="red">E2</font>.  
* [https://ascent.mpi-inf.mpg.de/ Ascent] from Max Planck.
* Ascent [https://ascent.mpi-inf.mpg.de/qa online answering demo]


The rest of the lecture will explain stuff in [https://github.com/tammet/nlpsolver nlpsolver]. See the [[Media:Nlpsolver_presentation.pdf|nlpsolver presentation]].


You may want to have a look at the AllenAI tutorial
Useful background:
* [https://github.com/allenai/acl2018-semantic-parsing-tutorial semantic-parsing-tutorial] with a lot of details and fancy stuff.


Additionally you may want to read some papers about semantic parsing:
About important concepts in parsing:
* Read a [https://www.aclweb.org/anthology/P15-1129.pdf paper] about building a semantic parser in a relatively simple way.
* [https://en.wikipedia.org/wiki/Part_of_speech Part-Of-Speech (POS)] and [https://en.wikipedia.org/wiki/Part-of-speech_tagging POS tagging]
* Another [https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/44858.pdf paper] about semantic parsing
* [https://web.stanford.edu/~jurafsky/slp3/18.pdf Jurafsky&Martin: Constituency Parsing]
* [https://web.stanford.edu/~jurafsky/slp3/19.pdf Jurafsky&Martin: Dependency Parsing]


There is an interesting specific method for semantic parsing, called AMR, worth having a brief look into:
Some important toolkits and representation schemes:
* [https://amr.isi.edu/language.html Amr site] with examples, papers etc.
* [https://spacy.io spacy], [[a small spacy UD parsing example]]
* [https://github.com/mdtux89/amr-eager A concrete implementation of an AMR parser]
* [https://stanfordnlp.github.io/stanza/ stanza]
* [https://paperswithcode.com/task/amr-parsing A catalogue of papers about AMR parsing]
* [https://universaldependencies.org/ universal dependencies]


There is a very good fairly deep NLP-and-commonsense series of podcasts and other interesting stuff from the [https://allenai.org/ Allen Institute for AI]:
Some draft parsing papers:
* [https://soundcloud.com/nlp-highlights NLP highlights] podcasts
* [[Media:concepts_draft_paper.pdf|concepts, based on UD]]
* [https://soundcloud.com/nlp-highlights/89-dialog-systems-with-zhou-yu episode 89] about dialog systems
* [[Media:pipeline_draft_paper.pdf|pipeline, based on UD]]
* [https://soundcloud.com/nlp-highlights/executable-semantic-parsing-with-jonathan-berant episode 91] about executable semantic parsing
* [[Media:llm_parser_draft_paper.pdf|experiments with LLM usage]]
* [https://soundcloud.com/nlp-highlights/59-weakly-supervised-semantic-parsing-with-abstract-examples-with-omer-goldman episode 59] about semantic parsing
* [https://allenai.org/demos demos], in partcular [https://rule-reasoning.apps.allenai.org/ Rover demo]


=== Lecture: research in hybrid systems: 18 aprill ===


[https://livettu.sharepoint.com/:v:/s/Knowledgerepresentation2023/EbB7A92RcW1Hul1VLQZ8vLMBh2NDteNwQ-L-p_9z1EJzSg?e=0QiAOz Teamsi salvestus 2023]
=== Lecture: we explore some recent papers in neurosymbolic reasoning: 27 April  ===


[[Hübriidsüsteemid: loengu käigus tehtud sissjeuhatavad märkmed]]
A reading session: we will read and attempt to understand together some case studies.


* Google DeepMind paper [https://www.nature.com/articles/s41586-023-06747-5 Solving olympiad geometry without human demonstrations]
* Google DeepMind paper [https://www.nature.com/articles/s41586-025-09833-y Olympiad-level formal mathematical reasoning with reinforcement learning]
* [https://arxiv.org/abs/2502.13834 Proving Olympiad Inequalities by Synergizing LLMs and Symbolic Reasoning]
* [https://arxiv.org/abs/2511.11751 Concept-RuleNet: Grounded Multi-Agent Neurosymbolic Reasoning in Vision Language Models]
* [https://proceedings.neurips.cc/paper_files/paper/2023/file/79fea214543ba263952ac3f4e5452b14-Paper-Conference.pdf What’s Left?Concept Grounding with Logic-Enhanced Foundation Models]


<!--
=== Lecture: visual reasoning: 4. May  ===
=== TPTP and the landscape of reasoners ===


* [http://www.cs.miami.edu/~tptp/ TPTP] problem set for first order logic
The lecture will be given by our postdoc researcher [https://vaishnavmohit.github.io/ Mohit Vaishnav].  
* See [[how to run gkc on the Wordnet TPTP version]]
* [http://www.cs.miami.edu/~tptp/CASC/ CASC prover competition]
-->


[[Media:Visual Reasoning 2026.pdf|Lecture presentation]] <font color="red">E2</font>


<!--
Related preprints:
=== Blocks world ===
* [https://arxiv.org/pdf/2501.13620 Cognitive paradigms for evaluating vlms on visual reasoning task]
 
* [https://arxiv.org/pdf/2604.21346 Symbolic Grounding Reveals Representational Bottlenecks in Abstract Visual Reasoning]
* [http://www.ee.pdx.edu/~mperkows/May4/005.Ch11-planning.ppt Old intro: planning and blocks world]


Some axiom sets:
=== Lecture: we continue exploring recent papers in neurosymbolic reasoning: 11 May  ===
* [[PLA001-0.ax]]
* [[PLA001-1.ax]]


And some planning problems to solve from these axiom sets:
We look at some of the more interesting papers from each subarea (maybe not all the papers):


* [[PLA004-1.p]]
Logic focus:
* [[PLA005-1.p]]
* Neuro-Symbolic Concepts https://arxiv.org/pdf/2505.06191
* [[PLA019-1.p]]
* Sound and Complete Neurosymbolic Reasoning with LLM-Grounded Interpretations https://arxiv.org/pdf/2507.09751


Also:
Visual reasoning:
* MuSLR: Multimodal Symbolic Logical Reasoning https://arxiv.org/pdf/2509.25851
* Can VLMs Reason Robustly? A Neuro-Symbolic Investigation https://arxiv.org/pdf/2603.23867


[[blocks world axiom/query examples from lecture|Alternative axiomatization]]
Planning:
-->
* LLMs as Planning Formalizers: A Survey for Leveraging Large Language Models to Construct Automated Planning Models https://arxiv.org/pdf/2503.18971
<!--
* Planning in the Dark: LLM-Symbolic Planning Pipeline without Experts https://arxiv.org/pdf/2409.15915 or newer https://ojs.aaai.org/index.php/AAAI/article/view/34855/37010
We use the [https://github.com/tammet/gkc gkc] prover for examples.
* Fast and Accurate Task Planning using Neuro-Symbolic Language Models and Multi-level Goal Decomposition https://arxiv.org/pdf/2409.19250
-->
* See also PDDL https://fareskalaboud.github.io/LearnPDDL/ https://homepages.inf.ed.ac.uk/mfourman/tools/propplan/pddl.pdf
<!--
Some background links:


** [http://www.cs.unm.edu/~mccune/papers/robbins/ Otter solving an open problem in math]
Text-to-SQL:
** [http://www.math.md/files/qrs/v10-n1/v10-n1-(pp95-114).pdf using otter for algebra problems]
* SPIDER 2.0: EVALUATING LANGUAGE MODELS ON REAL-WORLD ENTERPRISE TEXT-TO-SQL WORKFLOWS https://arxiv.org/pdf/2411.07763
** [http://www.cs.miami.edu/~tptp/ TPTP problem set for first order logic formalizations]
* CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQL https://arxiv.org/pdf/2410.01943
** [https://en.wikipedia.org/wiki/Zermelo%E2%80%93Fraenkel_set_theory set theory] and [http://www.cs.miami.edu/~tptp/cgi-bin/SeeTPTP?Category=Axioms&File=SET004-0.ax its axiomatization in logic] and [http://www.cs.miami.edu/~tptp/cgi-bin/SeeTPTP?Category=Problems&Domain=SET set theory problems in TPTP]
** [http://www.cs.miami.edu/~tptp/CASC/ CASC prover competition]
-->
<!--
=== Lecture: starting to parse natural language for interaction with rule reasoners ===


We will start building a tiny question-answering tool, translating natural language to rule reasoner input.
=== Consultation for the exam: 18 May  ===


Our first, very simple goal: translate the text "John is a father of Pete. Pete is a father of Mark. Who is the father of Pete?" to
a rule reasoner input
<pre>
father(john,pete).
father(pete,mark).
-father(X,pete) | ans(X).
</pre>
then run the reasoner, fetch the ans(john) and give an answer "John is".


You can use a trivial NLP-to-reasoner parser in python [[nlp.py]] as a starting point.


We will then proceed to gradually make questions and sentences more complex and add rules to be able to answer them.
<!--
 
[[Media:Research_in_Neurosymbolic_Reasoning.pdf|lecture slides]]
In the final part of the course we will start looking at uncertain reasoning: right now we will only try to do simple strict reasoning.
 
We will have a look at potential tools for semantic parsing, from simpler to more complex:
* [https://github.com/Kitt-AI/parsetron parsetron]: a really simple and very limited tool to build your own parser.
* [https://github.com/wcmac/sippycup sippycup]: has a detailed [https://nbviewer.jupyter.org/github/wcmac/sippycup/tree/master/ tutorial/course] attached, check the [https://nbviewer.jupyter.org/github/wcmac/sippycup/blob/master/sippycup-unit-0.ipynb first lab]
* [https://github.com/mynlp/ccg2lambda ccg2lambda]: parses to classical logic strings, which are embedded in XML output.


Other interesting things to look at:
The lecture will be given by our PhD student [https://aire-edih.eu/meeskond/otto-mattas/ Otto Mättas].
* [https://framenet.icsi.berkeley.edu/fndrupal/ Framenet]: map meaning to form in English through the theory of Frame Semantics.
* [https://github.com/google/sling Sling]: Google project for getting facts from Wikipedia using Frame Semantics.
* [https://yale-lily.github.io/spider Spider]: Yale Semantic Parsing and Text-to-SQL Challenge.
* [https://github.com/salesforce/WikiSQL WikiSQL]: A Salesforce crowd-sourced dataset for developing NLP interfaces for relational databases.
* [https://github.com/allenai/allennlp Allennlp] from the [https://allenai.org/ Allen Institute for AI]. See also their [https://github.com/allenai/acl2018-semantic-parsing-tutorial semantic parsing tutorial].
-->
-->
<!--
=== Lecture: visual reasoning  ===


=== Lecture: further details for semantic parsing: 25 aprill  ===
The lecture will be given by our postdoc researcher Mohit Vaishnav. You may want to have a look at his [https://vaishnavmohit.github.io/phd_thesis/ thesis] from 2023.


Loengu annab Martin: audikas pluss teamsis.
=== Lecture: third lab live experiments ===


[https://teams.microsoft.com/l/meetup-join/19%3apbCAi1dcW-0ZYLt9oaEWEF0OVSCh_itNLauu6apjLII1%40thread.tacv2/1682415298039?context=%7b%22Tid%22%3a%223efd4d88-9b88-4fc9-b6c0-c7ca50f1db57%22%2c%22Oid%22%3a%22e3e55dca-e89b-46ae-b8ee-1d158e79281d%22%7d Teamsi link 2023]
We will do live experiments for the third lab.


Loenguslaidid: [[Media:KR_2023_Parsing_Verrev.pdf]] <font color="red">E2</font>
Previous plan was: continue with uncertainty, numeric confidences and probabilities.


<!--
A [[Media:Confidences_cade28_d.pdf |presentation]] of our [https://link.springer.com/chapter/10.1007/978-3-030-79876-5_29 paper] about implementing numeric confidences. Here are [https://logictools.org/confer/ examples], a binary, etc.
[https://livettu.sharepoint.com/:v:/r/sites/Knowledgerepresentation2022/Shared%20Documents/General/Recordings/Teadmiste%20loeng%2026%20aprill_%20semantilised%20NLP%20parserid-20220426_100254-Meeting%20Recording.mp4?csf=1&web=1&e=m4kIuB Teams recording]
-->
-->
=== Reasoning with uncertainty: discrete reasoning options: 2. mai ===
[https://livettu.sharepoint.com/:v:/s/Knowledgerepresentation2023/EY0kmT4IweJIsIyItYzNHRgBfwEuNHeWnXpYJ1mWC130Dg?e=tHBC2b Loengusalvestus teamsis] 2023.
<!--
<!--
[https://livettu.sharepoint.com/:v:/r/sites/Knowledgerepresentation2022/Shared%20Documents/General/Recordings/Teadmiste%20loeng%203%20mai-20220503_100903-Meeting%20Recording.mp4?csf=1&web=1&e=99111g Lecture recording] in teams.
=== Lecture: continuing with uncertain knowledge ===
--
>
This week is devoted to discrete methods. Please read the materials described below slowly and carefully: the  <font color="red">E1</font> materials are what I expect you to know well at the exam and the  <font color="red">E2</font> materials are what you should know a bit at the exam.


Experimentation with the DLV system is recommended!
Wider background: [https://en.wikipedia.org/wiki/Non-monotonic_logic non-monotonic logic]


Roughly said, there are three kinds of discrete methods:
A [[Media:Ijcar_2022_tammet.pdf|presentation]] <font color="red">E2</font> of our [https://link.springer.com/chapter/10.1007/978-3-031-10769-6_18 paper] about implementing [https://en.wikipedia.org/wiki/Default_logic default logic] <font color="red">E1</font>. Here are [https://logictools.org/gk/ examples], a binary, etc.


* <b>Exceptions to rules</b>. The main method is called [http://en.wikipedia.org/wiki/Default_logic default logic]. <font color="red">E1</font> <b>Read carefully!</b>. Another, less popular method is [https://en.wikipedia.org/wiki/Circumscription_(logic) circumscription] (skim through, careful reading not needed). The main classical example for default logic is "birds normally fly": bird(X) & [we fail to prove -flies(X)] => flies(X). In other words, a rule is blocked, if we manage to prove that a negation of the rule-derived fact can be proved. Say we have
Additionally, we look at non-montonic logic using [https://swish.swi-prolog.org/example/scasp.swinb s(CASP)] and [https://www.dlvsystem.it/dlvsite/ dlv]. Both are examples of [https://en.wikipedia.org/wiki/Answer_set_programming Answer set programming (ASP)] and related to [https://en.wikipedia.org/wiki/Datalog datalog]. For ASP, have a look at [https://acris.aalto.fi/ws/portalfiles/portal/30223846/SCI_Janhunen_Niemela_The_Answer_Set_Programming.pdf this intro], written by [https://www.aalto.fi/en/people/ilkka-niemela Ilkka Niemelä] who is currently the [https://www.aalto.fi/en/aalto-university/governance-president president (rehtori) of the Aalto University]
** bird(tweety)
** penguin(pengu)
** (bird(X) & [we fail to prove -flies(X)]) => flies(X)
** penguin(X) => -flies(X)
:Then we can derive -flies(pengu) and flies(tweety) but we cannot derive flies(pengu), since it is blocked by -flies(pengu) which we could derive. In the wikipedia page above the "justification P" really means "we fail to prove -P".  
<!--
: Et asjadest paremini aru saada, vaata lisaks veidi [https://homepage.ruhr-uni-bochum.de/defeasible-reasoning/Courses/Natal2015-NonMonLog/natal-day-1-handout.pdf põhjalikumat eranditega järeldamise tutoriali] <font color="red">E2</font>
-->
-->


:Esimest järku loogikas ei ole üldjuhul võimalik välja rehkendada, et mingi asi <b>ei ole</b> järelduv. Lausearvutuses samas on. Sestap on olemas selline tore valdkond nagu [https://en.wikipedia.org/wiki/Answer_set_programming answer set programming], <font color="red">E2</font> kus esimest järku muutujatega valem teisendatakse lausearvutuseks, tekitades muutujatega reeglitest hästi palju konstante sisaldavaid reegleid. Siis muutuvad blokkerite arvutamised võimalikuks. Võid tahta lugeda [https://ojs.aaai.org/index.php/aimagazine/article/view/2671/2573 päris head introt] teemasse, mille on kirjutanud Aalto ülikooli rektor Niemelä. Samas ei ole see lausearvutuseks-teisendamine mittetriviaalsetel juhtudel võimalik. Üks paremaid süsteeme seal on [http://www.dlvsystem.com/ DLV] ([https://en.wikipedia.org/wiki/DLV DLV wikis]), vaata [https://www.dlvsystem.it/dlvsite/dlv-user-manual/ DLV manuaali]  Kui sul tekib rohkem huvi, siis selle answer-set programming lähenemise jaoks on olemas [https://teaching.potassco.org/ terve kursus] videote ja loengumaterjalidega.
<!--
=== Lecture: Semantic parsing ===


:Teine oluline küsimus eranditega reeglite puhul on nende prioriteedid: kui kaks reeglit üksteist blokivad, kas mõnel peaks olema kõrgem prioriteet? Näiteks, "materiaalsed asjad üldiselt ei lenda", "linnud on materiaalsed asjad", "linnud üldiselt lendavad", "pingiviinid on linnud", "pingviinid üldiselt ei lenda". Siin paistab, et taksonoomias allpool olevaid asju kirjeldavad reeglid võiks olla kõrgema prioriteediga kui ülalpool asju kirjeldavad reeglid. Samas "Nixoni kolmnurga" näites "vabariiklased ei ole patsifistid", "kveekerid on patsifistid", "Nixon korraga vabariiklane ja kveeker" oleks loomulik mõelda, et esimesed kaks default reeglit Nixoni jaoks blokeeruvadki ja me ei saa järelda, et Nixon on patsifist, ega ka seda, et ta ei ole patsifist. Prioriteetide teema kohta võib vaadata täeindavalt [https://www.ijcai.org/Proceedings/89-2/Papers/031.pdf seda artiklit] või [https://link.springer.com/content/pdf/10.1007/BF01535841.pdf seda sarnast artiklit].
The focus of the lecture is on [https://en.wikipedia.org/wiki/Semantic_parsing semantic parsing].


* <b>Describing beliefs</b> of various actors. This builds upon modal logic (see  [http://lambda.ee/w/images/2/20/Modaalsed_uuem.pdf intro presentation] <!-- <font color="red">E2</font> --> to understand the basics) and the main method is (auto)epistemic logic: the logic for encoding who knows or believes what. See the long tutorial [https://cs.nyu.edu/davise/knowledge-tutorial.pdf as an easily read text] <font color="red">E2</font> and additionally tutorial slides [[Media:Ijcai93.pdf|Ijcai93.pdf]] by the same authors to accompany the long text. <b>Read carefully</b>, starting with the long text and also looking at slides at the side.
Here are Martin's slides from 2024 [[Media:KR_2024_Parsing_01.pdf]] <font color="red">E2</font>.  
* <b>Multi-valued logics</b> with belief degrees (a la "very unlikely", "very likely") at different intervals. We will not look into these: we'll investigate the numeric methods instead. Have a very brief look at the [https://en.wikipedia.org/wiki/Many-valued_logic wikipedia].


NB! Modal beliefs and multivalued logics are not really used in any kinds of practical systems: they are hard to efficiently implement and they are not understood well enough in practical settings. However, different versions of default logics (and other systems with exceptions) are actually used, including in https://github.com/tammet/nlpsolver
We will look at both (a) classic paradigms of semantic parsing (as on the slides), (b) using LLMs and alternative modern paradigms for semantic parsing.


=== Reasoning with uncertainty: combining discrete and numeric + context: 9 mai ===
About important concepts in parsing:
* [https://en.wikipedia.org/wiki/Part_of_speech Part-Of-Speech (POS)] and [https://en.wikipedia.org/wiki/Part-of-speech_tagging POS tagging]
* [https://web.stanford.edu/~jurafsky/slp3/18.pdf Jurafsky&Martin: Constituency Parsing]
* [https://web.stanford.edu/~jurafsky/slp3/19.pdf Jurafsky&Martin: Dependency Parsing]


[https://livettu.sharepoint.com/:v:/s/Knowledgerepresentation2023/Ed1rou-hV5NPgstAfhyg4qABNdKgvhdv7Ek_kKCSmJWMOg?e=DCt1TI loengusalvestus 2023]
Some important toolkits and representation schemes:
 
* [https://spacy.io spacy], [[a small spacy UD parsing example]]
[https://link.springer.com/chapter/10.1007/978-3-031-10769-6_18 Artikkel] default reasoningu kohta.
* [https://stanfordnlp.github.io/stanza/ stanza]
 
* [https://universaldependencies.org/ universal dependencies]
[http://turing.cs.ttu.ee/~tanel.tammet/nlp_reasoning_pipeline.pdf Artikkel] kogu pipeline kohta: parsimine, reasoning, vastused. <font color="red">E2</font>


Some draft parsing papers:
* [[Media:concepts_draft_paper.pdf|concepts, based on UD]]
* [[Media:pipeline_draft_paper.pdf|pipeline, based on UD]]
* [[Media:llm_parser_draft_paper.pdf|experiments with LLM usage]]
-->
<!--
<!--
Read [http://dijkstra.cs.ttu.ee/~tammet/Implementing_Default_Logic.pdf Implementing_Default_Logic.pdf draft]<font color="red">E2</font>.
=== Semantic parsing continued  ===
-->


We will look at a number of small examples using both numeric and discrete
Lecture will be given by Martin Verrev on teams only (not the lecture room).
uncertainty, plus context, and will experiment with our new experimental solver in a live session.
<!--
* What is our approach to using default rules?
* How do we handle numeric uncertainty and default rules together?
* What works OK and what is problematic?
* What are some important practical issues to be solved?
* Some ideas for building a knowledge base with uncertainties.


See also [[draft notes for our approach to reasoning with defaults]]
Loenguslaidid (lisatud linkidega): [[Media:KR_2024_Parsing_02.pdf]] <font color="red">E2</font>
-->
-->
=== Konsultatsioon eksamiks: 16 mai===


<!--
<!--
[[Märkmed konteksti teemal]]
=== Topics in neurosymbolic reasoning and consultation for the exam  ===
-->
-->
<!-- [[Teadmiste formaliseerimise eksamiülesanded 20 mai 2020]] -->
<!--
<!--
== Block 7: context, time, location, events, causality. ==
Some papers to explore:


To be filled.
* https://ceur-ws.org/Vol-3432/
* https://ceur-ws.org/Vol-3432/paper5.pdf
* https://ceur-ws.org/Vol-3432/paper8.pdf
* https://aclanthology.org/2023.emnlp-main.313.pdf
* https://ojs.aaai.org/index.php/AAAI/article/view/29938/31639
* https://arxiv.org/pdf/2404.12698
* https://proceedings.neurips.cc/paper_files/paper/2023/file/79fea214543ba263952ac3f4e5452b14-Paper-Conference.pdf
* https://proceedings.neurips.cc/paper_files/paper/2023/file/ba90e56a74fd77d0ddec033dc199f0fa-Paper-Conference.pdf
-->
-->


<!--
 
[[Teadmiste formaliseerimise eksamiülesanded 20 mai 2020]]
[[Teadmiste formaliseerimise eksamiülesanded 27 mai 2020]]
-->
<!--
Other stuff: [[Commonsense rules development]]
-->
[[Category:Kursus]]
[[Category:Kursus]]

Viimane redaktsioon: 12. juuni 2026, kell 22:19

teadmised
teadmised

Name: Knowlege representation
Code: ITI8700
Link: http://lambda.ee/Teadmiste_formaliseerimine
Lecturer: Tanel Tammet tanel.tammet@ttu.ee ICT-426
Practice sessions: Riina Maigre
Archives of previous years: 2025.

Eksamitulemused

Eksamitulemused siin:

Eksamikonsultatsioon

(a) väike salvestuse teamsi, teemaks siis eksamikorraldus, ülesanded jne:salvestus siin, asub kursuse teamsi-salvestuste kataloogis.

(b) Konsultatsioonide lingid siin:

  • eksamikonsultatsioon 20. mai kell 10:30-11:00. (enne seda, kell 10:00 hajussüsteemide konsultatsioon, tolle kursuse lehel oleval lingil).
  • eksamikonsultatsioon 25. mai kell 10:30-11:00. (enne seda, kell 10:00 hajussüsteemide konsultatsioon, tolle kursuse lehel oleval lingil).

Eksamikorraldus ja materjalid õppimiseks

Eksamiks on kokku aega 2.5 tundi, aga enamik peaks saama hakkama ca 1.5 tunniga.

Eksam on kirjalik, samas ruumis/ajal toimub ka hajussüsteemide kirjalik eksam.

Allolevad punase E1-ga on eksami jaoks kõige olulisemad materjalid, E2 veidi vähem olulised. NB! Võrreldes varasemaga on lisaks markeeritud E2-ga need varasemalt olemas, aga markeerimata materjalid:

Ekamil tuleb neli eraldi hinnatavat osa: kolm teemaküsimust ja üks osa paari lühiküsimusega. Spetsiifiliste süntaksite (RDF, loogikavarandid jne) osas: ühtegi täpset süntaksit/võtmesõnu ei pea teadma, aga esitus peab olema õppejõule sisuliselt arusaadav. Millised on teemaküsimused ja millised on lühiküsimused:

SQL ja loogika: esitan paar tabelit ja küsin mõnda taolist küsimust:

  • nende sisu võimalikult täpset esitust esimest järku loogikas või RDFs
  • mõne konkreetse where- või join-päringu kirjutamist esimest järku loogikas

JSON-LD: annan JSON-LD jupi ja küsin selle tähendust ehk teisendust RDF kujule kolmikutena.

Reeglite esitamine: pane rdf ja/või rdfs kujule ja/või predikaatloogikale baseerudes kirja etteantud konkreetsed inimkeelsed väited.

Reeglite rakendamine klassikalises teoreemitõestamises: oskus teha käsitsi läbi väike resolutsioonimeetodi rakendamise näide, kui disjunktid ette antud.

Sõnavektorid ja N-grammid ja keelemudelite üldprintsiibid: arusaamine, oskus esitada väike näide töötamise kohta, võimalikud kasutus- või loomisstsenaariumd

Ebakindlate teadmiste esitamise põhimeetodid: pidevad (erinevat sorti tõenäosuste-tüüpi esitus) ja diskreetsed (eranditega süsteemid, eeskätt default logic). Siin võin tuua väikese näite ja küsida lahendust või palun ise esitada mõne pisinäite sobival kujul. Konkreetseid süntakseid teadma ei pea, piisab õppejõule arusaadavaks tegemisest.

Lühiküsimused (siit valitakse iga eksami jaoks paar):

Seleta paari lausega, mis asi on, vahel küsin ka näidet: freimiprobleem, open/closed world, blocks world, RAG, erinevad tõenäosuste interpretatsioonid, RDFa; mis asi on ja mida sealt umbes leida võib ja mida pigem mitte: schema.org, wordnet, conceptnet, yago, cyc; mida sa mäletad "semantiline parsimine" ja "visual reasoning" teemade kohta.

Eksamipäevad ja ajad

Siin on täpsustunud esimese kahe eksami ajad:

  • 27 mai, kolmapäev, kell 11:00 (koos hajussüsteemide eksamiga). Auditoorium NRG-226.
  • 1 juuni, esmaspäev: eraldi eksamiajad hajussüsteemide (11:00) ja teadmiste esitamise (14:00) jaoks. Auditoorium SOC-211a.
  • 8 juuni, esmaspäev: kell 11:00 (koos hajussüsteemide eksamiga). Auditoorium ICT-315.


Time, place, result

Lectures (Tanel): Mondays 12:00-13:30 room SOC 414
Practice sessions (Riina): Tuesdays 08:00-09:30 room MEK-122 MEK-109.

The lectures and sessions will be on site (physical room) plus teams. Later we may decide to do some lectures/practice sessions on teams only: this will be advertised on the page here early. Beware: recording may not always succeed.

Please join the teams group of the course. The code for joining is zb3z457.

Weekly teams link for the lecture.

Weekly teams link for the practice session.

Grading

Practical work will give 50%, exam 50% of points, and weekly tasks additional extra up to 10%, underlying the final grade. The exam will consist of (a) several questions asking for the explanation of important concepts along with examples (b) some small excercises.

The materials below marked with the red E1 are the main materials for the exam (10 of these on the page), while E2 materials (20 on the page) will also come up, but less and/or in a simpler form.

To finish the course successfully, you need to

  • successfully pass three practical works
  • receive at least 1/3 of the max points at exam
  • get practice work + weekly tasks + exam at least 50% points altogehter.

Contents overview: what the course is about

The theme of the course: from SQL to natural language.

Hence the main focus of the course is on hybrid methods for knowledge representation and reasoning: symbolic AI, machine learning / neural methods and their neurosymbolic combinations. We look at this spectrum from simple (representing and using knowledge in databases) to complex (meaning of sentences in natural language and commonsense knowledge) tasks. A closely related subject is commonsense reasoning.

The course contains the following blocks:

  • Knowledge in SQL, RDF and JSON
  • Handling simple facts and rules in symbolic AI and logic
  • General-knowledge databases
  • Natural language and question answering
  • Uncertain knowledge and hybrid systems

Weekly tasks

The small weekly tasks are in addition to the practical work: their goal is to practice with the stuff presented at the last lecture.

The first weekly task

Ülesanne on eksperimenteerida veidi logictools.org saidiga ja teha ise läbi loengus toodud näited SQL data ja päringute esitamisest loogikas. Tähtaeg 17. veebruar.

Konkreetselt tee järgmist:

  • Eksperimenteeri veidi logictools.org "simple examples" valikuboksi näidetega basic ... equality, loe nende selgitusi ja püüa aru saada tõestustest.
  • Loe seejuures predikaatloogika väikest seletust about lehel kuni lausearvutuseni (seda pole enam otse vaja lugeda).
  • Kirjuta ise valmis näide vastavalt kursuse materjalile Background: relations of sql and logic lk pealkirjaga "SQL join as logical rule", kus on toodud ühe client tabeliga ja client ning cars tabelitega näide sql päringute teisendamisest loogikasse (üks lihtne select ja üks join):
client table: id name balance
1 john 100 
2 pete -200

cars table: id model owner
1 ford 1
2 opel 2
3 saab 2

select client.name, client.balance
from client where balance<0

select client.name, cars.model from client, cars
where client.id=cars.owner

Tee need näited reaalselt valmis nii, et logictools.org sööb nad sisse ja annab ootuspärase vastuse. Seejuures on sul vaja kasutada $ans predikaati, nagu "answers" näites toodud. Kui tahad mitut vastust (ei ole kohustuslik), siis vaata ka "multiple answers" näidet.

  • Saada näite-sisendid ja genereeritud tõestused õppejõule emailiga (riina.maigre at taltech.ee ja cc tanel.tammet at taltech.ee). Kui jääd nendega hätta, siis kirjuta, mida proovisid, too oma näitetekst ja selgita, mille juures hätta jäid (see on ka arvestatav tulemus): kirjuta aadressile riina.maigre at taltech.ee või tanel.tammet at taltech.ee ja pane pealkirja sisse mh sõna "formaliseerimine", et kirja kergesti üles leiaks.
  • Kasuta kindlasti oma taltech.ee aadressi, sest muidu võib kiri lõpetada rämpspostis!

Soovitused:

  • Negatiivset arvu sa logictoolsi vaikimisi süntaksiga otse sisestada ei saa: -200 interpreteeritakse kui loogilise eituse rakendamist 200-le. Kaks head võimalust negatiivse arvu sisestamiseks on kas kirjutada 0-200 või $difference(0,200).
  • Aritmeetika jaoks vaata help teksti Arithmetic peatükki.
  • Väiksem-tingimuse jaoks vaata complex examples valikuboksis "arithmetic" näidet.


The second weekly task

Selle ülesande mõte on veidi praktiseerida JSON-LD esitusega. Sinu konkreetne ülesanne on võtta oma esimene nädalaülesanne ja esitada needsamad andmed (ja soovitavalt ka reeglid) JSON-LD kujul.

Vaata alustuseks json-ld näiteid logictoolsi json-tabi alt. NB! Enamus näiteid "example" valikuboksist ei ole konventsionaalne json-ld, aga mitmed on.


Practical work

There are three labs.

The labs have to be presented (as a demo + small overview of the software and principles) to the course teachers and all students present at labwork time, either in the auditorium or over MS teams. The labs can be prepared alone or by teams of two or three people.

Each lab is evaluated independently and gives the same number of points. NB! When you do not meet the lab deadline, you will be awarded half of the points for this lab.

First lab

The 2026 task for the first lab is about encoding nontrivial information in various ways (standard logic, rdf, json-ld) and writing simple logical rules for answering questions.

Deadline: 10. March.

Second lab

The 2026 task for the second lab is about using a VLM (visual language model) to get a structured description from an image which would fit with the pre-existing simple world model.

Deadline: 7. April

Third lab: questions in NLP

This is the task for 2026:

Experiment with neurosymbolic reasoning: LLM as a parser plus a symbolic reasoner for images: 2026

Deadline: 12. May


Books to use

Main materials:

Other useful books and tools for symbolic K&R:

  • logictools.org for experimenting with classical automated reasoning
  • Symbolic reasoning themes are covered in this book with (hopefully) freely accessible pdf-s via Taltech. See also slides for the book.
  • Similarly, the freely accessible pdf of the handbook of knowledge representation gives a detailed and thorough coverage of the subject; far more than necessary for the course.
  • You may want to get a small intro to Prolog by reading the first 100 pages of a classic Art of Prolog book. It is not strictly necessary for the course (we will not be using Prolog) but it surely helps. Besides, the book is interesting, quite easy to read and works out a lot of interesting examples.

Other useful material for neural NLP:

Observe that a noticeable part of the course contents are not covered by these books: use the course materials and links to papers, standards and tutorials provided.

Blocks and lectures: the main content of the course

The details and materials of lectures in the future are from 2025 and will be revised/extended. The materials of the current or passed lectures are up to date.


Block 1: knowledge in SQL, RDF and JSON

Lecture: Intro and background: 2. February

Intro presentation:

teadmised_intro2026.pdf

Recommended non-obligatory listening: Francois Chollet, Yann LeCun, Gary Marcus, Demis Hassabis, Ilya Sutskever, Ben Goertzel ,

Lecture: Nosql, json and rdf in databases: 9. February

Lecture materials:

See also

The weekly homework will be about trying out small things in logic: please try out logictools from "basics" to "equality" in simple examples selectbox. Details will appear after the lecture.

Block 2: handling simple facts and rules in symbolic AI and logic

Lecture: simple rules in rdfs, logic and owl + reasoner: 16. February

Rdf example developed during lectureE2

Have a brief a look at this, no need to read thoroughly:

The main materials covered in the lecture:


And then have brief look (skim through) at:

Certainly read:

We will also start at looking and understanding the examples in logictools.org

More resources:

What the reasoner does:

We will consider the reasoning part of K&R to understand what provers do:

You may want to have a look at the additional materials - from easier to harder - from the

Lecture: DPP ontology development. 2. March

Loengu annab Riina. Digital Product Passport (DPP) kui näide praktilisest ontoloogia/sõnastiku arendamisest CIRPASS projektis.

Hetkeseisu algusleht.

CIRPASS-2

Ühikute ontoloogia

SHACL

UNTP DPP



Lecture: Next homework and looking into main large knowledge bases: 9. March

We will first present the next homework.

Then we will continue with existing large knowledge bases.

Slides:lecture slides E2, a tool to explore Wordnet taxonomies can be found here: https://github.com/martinve/wntool

You can search/investigate what kind of data is available and you can find out actual data from these databases with some work, by surfing on the web pages of the systems.

About knowlege base creation using LLMs: have a look at this paper.

We will have a look at the goals, main content and differences between:


Block 3: natural language and question answering

Lecture: Intro to NLP, n-grams, word vectors: 16 March

Lecture: large language models (LLM) 23 March


Eesmärk: saada pinnaliselt aru LLM ja transformerite (NLP masinõppe põhiasjad) peamistest ideedest. Detailid on tegelikult keerulised ja nendest arusaamine võtab hulga rohkem aega, kui meil kursuses on, sestap lepime vähesega. Samas, kui sul endal huvi, siis loomulikult on teretulnud detailsem iseõpe.

Enne järgmisse materjali süvenemist tutvu kindlasti vektorsemantikaga, mida seletasime 11 märtsi loengus.

Mida vaadata ja lugeda:

Eriti soovitav

Paratamatult jääb enamus detaile loengus vähearusaadavaks (kui ise ei kuluta hulga aega järele katsetamiseks), aga sellegipoolest on see tõenäoliselt kõige parem loeng transformeritest: mingi tunnetuse sellest, et mis toimub, saab.

Pikemaks lugemiseks:


Optsionaalselt üldisem arusaam klassikutelt:

Optsionaalselt diipim arusaam:

Lecture: RAG usage pattern / context engineering for LLMs : 30 March

Lecture material: this is focused on RAG and written in 2024 and spring 2025: good tool use, good reasoning and more autonomy like claude code and openclaw are not covered in this overview.

"Context Engineering" became a popular phrase in 2025: essentially it means "fancy RAG", i.e. significant effort put towards creating a really good set of additional information (prompt additions: "context") for each user query/prompt. It can also mean approaches for storing such additional information on the fly in special files which are then used RAG-like as extra input in addition to user dialog.For example, claude code memory principles and memory folder structure, openclaw memory.

Useful reading in addition to pointers in the lecture material:

Block 4: hybrid systems and uncertain knowledge

We will focus on (a) world modeling and uncertainty in KR (open/closed world, frame problem, probabilities, exceptions) and (b) current research on building hybrid systems

Lecture: issues in formalizing commonsense reasoning: open/closed world, frame problem, uncertainties: 6. April

First, open/closed world, world states, frame problem

The standard example for these is a blocks world: some blocks are on the table and the robot arm can lift single blocks and put them on other blocks or on the table. The arm cannot lift several blocks or a tower of blocks. Then, given a position of blocks at initial situation, can the robot arm create a required new position of blocks? Can we get the steps required? For getting the steps we use the $ans predicate.

One hard issue arising is that how do we know that doing some action like lifting a block does not create side effects like stumbling existing towers, moving other blocks etc? This issue is called the frame problem and it has no easy solutions.

Importantly, the frame problem arises since normal first order logic has

  • the open world assumption: if we do not have (and cannot derive) a positive fact like P(a) and neither a negative fact like -P(a), we assume we simply do not know which holds.

Prolog and databases operate under the

  • closed world assumption E1 (must read): if we do not have (and cannot derive) a positive fact like P(a), we automatically assume that -P(a) must be true. For example, Prolog "negation" really means "cannot prove". One consequence of this assumption is that we cannot represent real negation and disjunction in an OK manner (Prolog does not contain these) and cannot satisfactorily speak about "true", "false" and "unknown". See also negation as failure in prolog


Please look at and read these materials about the frame problem:

These two readings are optional:

  • frame problem at the Stanford Encyclopedia of philosophy: not obligatory, but a good read to get a better understanding.
  • another classic about frame problem: read this only if you became really really interested about the whole issue: it goes quite deep, although it is not especially technical.

You may also want to have a look at the algorithmic complexity and efficient solution algorithms, regardless of how we formalize the problem: see this article.

Next, importantly, you should experiment yourself with gkc and both of the following files. Please copy and read the files and understand the encoding. At the end of the files are several queries to experiment with: instructions are also there. The simplest way to experiment is to use https://logictools.org/ . There is already a blocks world example available.

In both of them the predicate holds(X,Y) is used to describe something X about the blocks world (like, one block is on another or robot holds some block) and the second argument Y describes the state we have arrived to via robot actions from the original state (like, picked up a block in the initial state and then did put the block down on top of another).

To get a simple visualization of the blocks world, have a look at this tiny video.


Second, uncertainty


Loengu sissejuhatava osa presentatsioon: Reasoning_with_uncertainty.pdf E1.

Almost all the knowledge we have is uncertain: there are many exceptions to a rule or a fact/rule holds with some vague probability. Notice that, in contrast, typical databases in companies contain facts with very high certainty, but they do not contain rule or commonsense knowledge: the latter is built into brittle software code of systems using these databases.

We will consider two different ways to tackle uncertainty:

  • Numeric methods: uncertainty is estimated with probability-like numbers of various kinds.
  • Discrete methods: no numbers used. Instead, uncertainty is described as exceptions or beliefs of people.

NB! Both of these ways are hard to actually implement, neither have they been understood very well, despite an immense amount of research and papers. For example, typical machine learning methods operate with extremely vague probability-like numbers and do not attempt to put these numbers into a real properly-theoretical-and-correct probabilities framework.

The main material for reading about numeric methods is an overview-followed-in-depth by the lecturer Numeric uncertainty: (E2 up to and including the chapter 3, see also E2 tags inside)

  • Read carefully up to and including the chapter 4 "Different ways to encode confidences in logic" and have a quick look at all the wikipedia links. In particular, Wiki on Bayesian inference and this
  • Skim through the later parts and look only into these subchapters which you found interesting.

Next,

  • Read the paper E2 about confidences. containing intro, algorithms and experiments. Then look at the web page with the confer reasoner and a wealth of examples.
  • Read the paper E2 about exceptions (default logic). containing intro, alorithms and experiments. Then look at the web page with the gk reasoner and a wealth of examples.

Next, have a brief look at a good example system doing some specific kinds of probability reasoning:

Lecture: research in neurosymbolic reasoning: 13 April

lecture slides

The lecture will be given by our PhD student Otto Mättas.

Lecture: semantic parsing with LLMs and with UDP: 20 April

The focus of the lecture is on semantic parsing.

For a quick intro we will use Martin's slides from 2024 semantic parsing E2.

The rest of the lecture will explain stuff in nlpsolver. See the nlpsolver presentation.

Useful background:

About important concepts in parsing:

Some important toolkits and representation schemes:

Some draft parsing papers:


Lecture: we explore some recent papers in neurosymbolic reasoning: 27 April

A reading session: we will read and attempt to understand together some case studies.

Lecture: visual reasoning: 4. May

The lecture will be given by our postdoc researcher Mohit Vaishnav.

Lecture presentation E2

Related preprints:

Lecture: we continue exploring recent papers in neurosymbolic reasoning: 11 May

We look at some of the more interesting papers from each subarea (maybe not all the papers):

Logic focus:

Visual reasoning:

Planning:

Text-to-SQL:

Consultation for the exam: 18 May

Pärit leheküljelt "http://www.lambda.ee/w/index.php?title=Teadmiste_formaliseerimine&oldid=23024"