Teadmiste formaliseerimine

teadmised

Name: Knowlege representation Code: ITI8700 Link: http://lambda.ee/Teadmiste_formaliseerimine Lecturer: Tanel Tammet Contact: tanel.tammet@ttu.ee, 6203457, TTÜ ICT-426 Archives of previous years: 2022.

Time, place, result

Lectures: Tuesday 13:45-15:15 room U05-103
Practical work: Tuesday 15:30-17:00 room U05-103

Plaan on teha osa loengutest kohapeal (esialgu kindlasti kohapeal) ja osa distantsilt. Loenguid salvestame ja salvestuste lingid on allpool blokkide/loengute juures.

Praksidega samamoodi: algul kindlasti kohapeal, samas üritame salvestada / üle kanda, edaspidi tõenäoliselt osa prakse distantsilt (teamsis).

Salvestuste lingid leiad allpool kas vastava loengu või praksi peatüki all.

Moodles materjale ei ole: kogu info, materjalid ja ülesanded on lambda lehel.

Practical work will give 40%, weekly tasks 10% and exam 50% of points underlying the final grade. The exam will consist of several small excercises.

To finish the course successfully, you need to

• successfully pass three practical works
• receive at least 1/3 of the max points at exam
• get practice work + weekly tasks + exam at least 50% points altogehter.

Contents overview: what the course is about

The main focus of the course is on hybrid methods for knowledge representation and reasoning: symbolic AI, machine learning / neural methods and their combinations. We look at this spectrum from simple (representing and using knowledge in databases) to complex (meaning of sentences in natural language and commonsense knowledge) tasks.

The course contains the following blocks:

• Representing knowledge in the SQL and document databases and hybrids.
• Representing and using simple facts and rules in symbolic AI and logic.
• General-knowledge databases: wikidata, wordnet, yago, conceptnet, nell, cyc.
• Knowledge conveyed by natural sentences: statistical and neural methods like word2vec, BERT, GPT families.
• Knowledge conveyed by natural sentences: logic-based and hybrid methods.
• More about reasoners and question answering systems.
• Confidence and relevance.
• Context, time, location, events, causality.

Check out this description of the whole KR area.

See also the intro lecture Media:Knowledge_Representation_Intro.pptx

Links to the teams group and lecture videos

The links to saved lecture videos are and will be visible among the materials of corresponding lectures below.

NB! In case you run into trouble (cannot join or no sound etc) please write via fb messenger or email tanel.tammet at taltech.ee.

Weekly tasks

These are small tasks you have to report weekly by email to tanel.tammet at taltech.ee.

Practical work

There are three labs. They are all steps in a single project: build a simple natural-language question-answering system a la classic Watson. There is also an alternative option to do one larger practical research project instead of these three labs: details of this project have to be discussed and agreed with Tanel.

The labs have to be presented (as a demo + small overview of the software and principles) to the course teachers and all students present at labwork time, either in the auditorium or over MS teams. The labs can be prepared alone or by teams of two or three people.

Each lab is evaluated independently and gives the same number of points. NB! When you do not meet the lab deadline, you will be awarded half of the points for this lab.

First lab

Please read about the task for the first lab.

Second lab

The task will be updated for 2023. The task from last year, 2022 was this:

• write a small set of additonal, nontrivial rules for geography not present in Yago or Wordnet,
• actually answering simple questions using logic and rules with the help of a reasoner.

Deadline: 6. April 2022.

See Details of the KR second lab in 2022

Third lab: numeric confidences or questions in NLP

This task will be updated for 2023. In 2022 you had two different choices for the third lab:

• Add and use uncertainty in rules (suggested)
• to convert English questions to logical queries (harder)

Details under the separate pages linked to above. A brief intro will be given during the lectures.

Alternative to labs: a research project

The goal of the research project is to build a good database of similar words along with the main differences between word/similar word. A la "queen" is similar to "king" except for gender etc. There are a number of papers explaining different methods. Tanel has built a prototype database/software for doing this from Wikipedia texts + NLP tools + Wordnet, but this prototype is too weak and also does not find main differences. The prototype can be used as a starting point.

Books to use

The main recommendations to get up to speed with very useful background:

• Get the background about automated reasoning from the book chapter pages 297-345 by Tanel Tammet and optionally more details from the set of course notes from Geoff Sutcliffe.
• Recommended background for NLP: Speech and Language Processing by Dan Jurafsky and James H. Martin. here is the 2nd edition and (suggested) here the web page for the draft 3rd edition. Recommendation: read from the beginning up to an including chapter 6: Vector Semantics and Embeddings, then skip and read a few chapters starting from 15: Logical Representations of Sentence Meaning. You do not need to understand all the details of math, for example. It is OK if you understand the main principles of representations and algorithms.

Other useful books:

• Ca half of the course themes are covered in this book with freely accessible pdf-s.
• The freely accessible pdf of the handbook of knowledge representation gives a detailed and thorough coverage of the subject; far more than necessary for the course.
• You may want to get a small intro to Prolog by reading the first 100 pages of a classic Art of Prolog book: pdf is freely downloadable under the "Open Access" tab. It is not strictly necessary for the course (we will not be using Prolog) but it surely helps. Besides, the book is interesting, quite easy to read and works out a lot of interesting examples.

Observe that a noticeable part of the course contents are not covered by these books: use the course materials and links to papers, standards and tutorials provided.

Blocks and lectures: the main content of the course

NB! Allpool olev blokkide struktuur on põhiliselt sama, kui 2021 aastal. Toimunud loengute materjalid ja salvestused osaliselt uuendatakse/muudetakse: kogu info veel mitte toimunud loengute kohta on esialgne. Ülesanded ja praksid, välja arvatud teine praks, on up to date ja mõeldud tegemiseks :)

Block 1: knowledge in the SQL and document databases and hybrids.

Lecture: Intro and background, 25. Jan

Esimese loengu salvestamine ebaõnnestus :( järgmine kord teeme paremini. Aga, palun (a) vaata 2021 sissejuhatavat loengut. (b) vaata üle 2022 tegelikult kasutatud sissejuhatuse presentatsioon:

Media:Knowledge_Representation_Intro.pptx

Soovitav (mitte kohustuslik) kuulamine:

• Marcus episode on the Lex Fridman podcast https://lexfridman.com/gary-marcus/
• Gary Marcus (KR&R commonsense proponent) & Joshua Bengio (ML proponent) https://www.facebook.com/watch/live/?v=498403850881660&ref=watch_permalink read also the related Marcus paper https://arxiv.org/ftp/arxiv/papers/1801/1801.00631.pdf
• David Ferrucci (IBM Watson team leader) on the Lex Fridman podcast https://lexfridman.com/david-ferrucci/
• Tim Rocktäschel (Facebook A.I.) episode https://towardsdatascience.com/language-models-symbolic-learning-and-the-road-to-agi-75725985cdf7
• Reading, not listening: on Hybrid A.I. in image analysis https://knowablemagazine.org/article/technology/2020/what-is-neurosymbolic-ai

Lecture: Nosql, json and rdf in databases: 1. Feb

Lecture materials:

• We cover Background: relations of sql and logic E1

• and start with schemaless databases, RDF and RDFS E1 (also important for practice work)

• Lecture recording in teams

See also

• Json in Postgresql: json datatype and functions for json
• notes about schema for rdf / first lab

The weekly homework will be about trying out small things in logic: please try out logictools from "basics" to "equality" in simple examples selectbox. Details will appear after the lecture.

Block 2: simple facts and rules in symbolic AI and logic

Lecture: simple rules in rdfs, logic and owl: 8 feb

• Main lecture material: schemaless databases, RDF and RDFS looked at rdfs rule meanings in logic and sparql (up to page 30 in the presentation).
• Lecture recording in echo.

Have a brief a look at this, no need to read thoroughly:

• Official w3c rdf and rdfs primer E2

Lecture: rdfa, json-ld, schema.org: 15 feb

Teamsi salvestuse link.

The main theme is different ways of presenting triple/rdf data

We will first have a look at the following to recall the material from the previous lecture:

• Official w3c rdf and rdfs primer E2

The main materials covered in the lecture:

• json-ld (check out playground examples) E1 currently most popular triple representation language on top of json. see also wikipedia and w3c standard: for the latter, read the basic concepts chapter E2.
• rdfa E2 triple markup language suited for html, see also wikipedia
• schema.org (E2 you can surf on the schema.org site and understand/find answers there)) we looked at in the last lecture: property markup vocabulary suggested by Google, Microsoft and others.

And then have brief look (skim through) at:

• Official w3c owl primer but it is better to look at this intro presentation and continue with the owl rules presentation
• sparql tutorial and wikipedia take on sparql.

Certainly read:

• Hea jutt Whatever Happened to the Semantic Web?

We will also start at looking and understanding the examples in logictools.org

More resources:

• sparql implementations in wikipedia
• various rdf tools listed at the bottom of the page

Lecture: rules in logic with provers to answer questions: 22 Feb

Loengusalvestus teamsis.

Recall and go over OWL ideas:

• Official w3c owl primer but it is better to look at this intro presentation and continue with the owl rules presentation. If you want, have a deeper look at the full book.

We will consider the reasoning part of K&R to understand what provers do:

• rules, logic, provers E1
• logictools system running gkc reasoner in the browser: try out the "simple" and "complex" examples, read explanatory texts and the manual about predicate logic
• Have a look at the gkc github repo, including precompiled binaries for command line in the latest release and the tutorial Examples/Readme.md.
• Travel examples from lecture 2021 in 2021.

Lecture: what the reasoner does: 1 March

Teamsi loengusalvestus

We will continue with the same presentation:

• rules, logic, provers

along with doing/explaining experiments using logictools.

We will also present the main details of the second practice work.

You may want to have a look at the additional materials - from easier to harder - from the book chapter pages 297-345 by Tanel Tammet, the the set of course notes from Geoff Sutcliffe, the implementation presentation from Stephan Schulz and the readable-and-hackable PyRes reasoner for educational use.

Block 3: General-knowledge databases

Lecture: looking into main large knowledge bases. 8. March

Loengusalvestus teamsis.

E2: you can search/investigate what kind of data is available and you can find out actual data from these databases with some work, by surfing on the web pages of the systems.

We will have a look at the goals, main content and differences between:

• wordnet see tptp axioms
• dbpedia, see also classes and dbpedia wiki
• wikidata
• yago old page and new page
• babelnet described in wikipedia
• conceptnet described in wikipedia
• nell website (sometimes down) and in wiki and a good paper.
• framenet described in wikipedia, see example
• cyc but just read wikipedia and see whitepaper also tptp small random selection of axioms
• adimen-sumo see sumo in wikipedia and tptp axioms
• tptp: a large set of axioms and problems in logic usable for automated reasoners
• schema.org: property markup vocabulary suggested by Google, Microsoft and others.

Lecture: Allen Ai research systems and Google, Facebook annotation systems: 15 märts

Link salvestusele.

Examples from Allen Ai research:

• Macaw demo: a predictor like GPT-3
• Proofwriter demo: a hybrid system (rules + predictor) (see the paper)
• Rule Taker demo: an earlier version of the Proofwriter above
• Mosaic commonsense reasoning
  • SWAG benchmark dataset
  • Atomic data
  • Atomic graph explorer
  • Atomic paper

Google, Facebook annotation systems:

• Google structured data E2 on the webpage. See also Google knowledge graph E2 in wikipedia.
• Facebook Open Graph markup on the webpage. See also https://ogp.me/ and the history of NLP-based ogp search

As a background, recall:

• json-ld (check out playground examples) E1 currently most popular triple representation language on top of json. see also wikipedia and w3c standard: for the latter, read the basic concepts chapter E2.
• rdfa E2 triple markup language suited for html, see also wikipedia
• schema.org (E2 you can surf on the schema.org site and understand/find answers there)) we looked at in the last lecture: property markup vocabulary suggested by Google, Microsoft and others.

Block 4: natural language and question answering

Lecture: Intro to NLP: 22 March

Loengu teamsi salvestus ja NLP sissejuhatuse märkmed loengust.

• These three wikipages give useful introductory details:Natural language processing, Knowledge extraction, Natural language understanding.
• Väga hea õpik: 3. variandi draft või 2. variandi terve pdf.
  • Hea mõte on hakatuseks lugeda läbi 2. variandi sissejuhatus.
  • Vektorsemantikast arusaamiseks loe 3. variandi ptk 3. N-gram Language Models ja siis chapter 6: Vector Semantics and Embeddings E2. Vektorsemantika kasutamine närvivõrkudes a la Bert ja GPT on hästi kirjas ptks 7: Neural Networks and Neural Language Models
• GPT-3 API beta publicly available without a waiting period. There is also a playground on the web.
• Kui tahad katsetada toore Wikipedia dataga, siis saab loengus vaadatud töödeldud variante siit (tarballid sisaldavad datat, selgitavat README-d ja tarkvara selle ise ehitamiseks): a compacted pure-text version of full wikipedia, A lemmatized version of wikipedia texts, Several co-occurrence matrices and lists of top-co-occurring words for wikipedia
• Online demo of NTLK running some basic NLP tasks

Recommended listening:

• A cool 14-minute philosophy podcast episode to listen: Jonathan Webber on deceiving with words plus a a direct audio link (may expire at some point)
• NLP highlights podcast series from the Allen institute for AI. Makes sense to start from the very first ones and pick these which have a title looking interesting to you. World-leading stuff and quite understandable.

Lecture: vector representation of words: 29. March

Loengusalvestus teamsis.

Nädala eesmärk on konkreetne: sul on vaja aru saada vektorsemantikast, ehk, mis see on, kuidas sõnavektoreid moodustatakse ja kuidas neid kasutada saab.

Now, you may have had a course on NLP and machine learning, in which case you should already know a lot about vector semantics. If so, consider yourself lucky :) and just skim through the following materials to check whether there are any new things or old friends for you. If not so, consider yourself extra lucky: you have come to the core of machine-learning-based NLP, and this is both pretty cool and quite understandable at the same time! Most of the machine-learning-based NLP is built on top of the ideas of vector representation of words.

As a continuation of the lecture from the last week, please have a look and try out

• Allen NLP demos for a number of core NLP techniques: highly recommended for experimentation.

Obligatory study materials:

• First, listen to text vectorization episode from the linear digressions podcast.
• Second, listen to word2vec episode from the linear digressions podcast.
• Finally the main thing: slowly read the whole Vector Semantics: chapter 6 E2 from the Jurafsky & Martin book.

Additional supporting non-obligatory materials from (roughly) easier to more complex:

• vector semantics part I presentation from the Jurafsky & Martin book for for chapter 6.
• vector representation: a short tutorial with code examples.
• word2vec tutorial from tensorflow
• vector semantics part II E2 presentation from the Jurafsky & Martin book for chapter 6.

If you want to get a deeper understanding and look at longer lectures while doing that, there is a wealth of good word vector lecture videos available, for example, from the Stanford NLP course

• Word vectors 1
• Word vectors 2
• Word vectors 3

However, I'd recommend to use the obligatory and supporting materials above first and only - optionally - spend time with the longer lectures after that.

Also interesting to look and read:

• Probabilistic models: LDA video explanation by Jordan Boyd-Graber
• Aetherial Symbols by the ultimate machine learning guru aka "godfather of machine learning" Geoffrey Hinton
• Autocomplete using Markov chains

Block 5: Conceptually complex issues in KR&R

We will focus on several conceptually complex issues in KR: open/closed world, frame problem, reasoning with uncertainty, semantic parsing of NLP, context etc.

All of these are critical for commonsense reasoning and none have really good answers or implementations so far.

Lecture: open/closed world, planning, frame problem, blocks world: 5. April

Loengusalvestus teamsis

For the main part of the lecture we consider planning and the frame problem. The standard example for these is a blocks world: some blocks are on the table and the robot arm can lift single blocks and put them on other blocks or on the table. The arm cannot lift several blocks or a tower of blocks. Then, given a position of blocks at initial situation, can the robot arm create a required new position of blocks? Can we get the steps required? For getting the steps we use the $ans predicate.

One hard issue arising is that how do we know that doing some action like lifting a block does not create side effects like stumbling existing towers, moving other blocks etc? This issue is called the frame problem and it has no easy solutions.

Importantly, the frame problem arises since normal first order logic has

• the open world assumption E1 (must read): if we do not have (and cannot derive) a positive fact like P(a) and neither a negative fact like -P(a), we assume we simply do not know which holds.

Prolog and databases operate under the

• closed world assumption E1 (must read): if we do not have (and cannot derive) a positive fact like P(a), we automatically assume that -P(a) must be true. For example, Prolog "negation" really means "cannot prove". One consequence of this assumption is that we cannot represent real negation and disjunction in an OK manner (Prolog does not contain these) and cannot satisfactorily speak about "true", "false" and "unknown". See also negation as failure in prolog

Please look at and read these materials about the frame problem:

• tiny video of the Boston Dynamics robot failing: looks like a frame problem in the blocks world :)
• overview presentation: E1 must read.
• a classic AI philosophy paper about the frame problem: must read the first three pages (rest is optional).
• wikipedia about frame problem: skim it through (no need to read all the details) to get a rough idea about different approaches to the frame problem. None of these are really good.

These two readings are optional:

• frame problem at the Stanford Encyclopedia of philosophy: not obligatory, but a good read to get a better understanding.
• another classic about frame problem: read this only if you became really really interested about the whole issue: it goes quite deep, although it is not especially technical.

You may also want to have a look at the algorithmic complexity and efficient solution algorithms, regardless of how we formalize the problem: see this article.

Next, importantly, you should experiment yourself with gkc and both of the following files. Please copy and read the files and understand the encoding. At the end of the files are several queries to experiment with: instructions are also there.

In both of them the predicate holds(X,Y) is used to describe something X about the blocks world (like, one block is on another or robot holds some block) and the second argument Y describes the state we have arrived to via robot actions from the original state (like, picked up a block in the initial state and then did put the block down on top of another).

• Simple axiomatization of the blocks world: E2 please also read through the comments explaining the axiomatization and queries!
• More complex blocks world axiomatization taken from the TPTP problem set for first order logic, concretely by concatenanting and commenting axiom sets PLA001-0.ax, PLA001-1.ax and queries PLA004-1.p, PLA005-1.p, PLA019-1.p

To get a simple visualization of the blocks world, have a look at this tiny video.

Reasoning with uncertainty: 12 April

Loengusalvestus.

Loengu sissejuhatava osa presentatsioon: Reasoning_with_uncertainty.pdf E1.

Almost all the knowledge we have is uncertain: there are many exceptions to a rule or a fact/rule holds with some vague probability. Notice that, in contrast, typical databases in companies contain facts with very high certainty, but they do not contain rule or commonsense knowledge: the latter is built into brittle software code of systems using these databases.

We will consider two different ways to tackle uncertainty:

• Numeric methods: uncertainty is estimated with probability-like numbers of various kinds.
• Discrete methods: no numbers used. Instead, uncertainty is described as exceptions or beliefs of people.

NB! Both of these ways are hard to actually implement, neither have they been understood very well, despite an immense amount of research and papers. For example, typical machine learning methods operate with extremely vague probability-like numbers and do not attempt to put these numbers into a real properly-theoretical-and-correct probabilities framework.

The main material for reading about numeric methods is an overview-followed-in-depth by the lecturer Numeric uncertainty: (E1 up to and including the chapter 4, see also E2 tags inside)

• Read carefully up to and including the chapter 4 "Different ways to encode confidences in logic" and have a quick look at all the wikipedia links.
• Skim through the later parts and look only into these subchapters which you found interesting.

Next,

• Read the paper E2. containing intro, alorithms and experiments.
• Look at the web page with the confer reasoner and a wealth of examples.

Next, have a brief look at a good example system doing some specific kinds of probability reasoning:

• Problog start with a tutorial E2
• Try out some examples via browser from the tutorial

Lecture: semantic parsing and further details for the third practical work: 19 April

Lecture recording.

Additionally, please listen and read in this order:

• Read through the convert English questions to logical queries
• Listen to the episode David Ferrucci: IBM Watson, Jeopardy & Deep Conversations with AI in a very popular ai podcast.
• Read a good presentation E1 about a history of semantic parsing: this won't tell you much about how to do it, though.

Also, check out a very useful demo:

• Ascent from Max Planck.
• Ascent online answering demo

You may want to have a look at the AllenAI tutorial

• semantic-parsing-tutorial with a lot of details and fancy stuff.

Additionally you may want to read some papers about semantic parsing:

• Read a paper about building a semantic parser in a relatively simple way.
• Another paper about semantic parsing

There is an interesting specific method for semantic parsing, called AMR, worth having a brief look into:

• Amr site with examples, papers etc.
• A concrete implementation of an AMR parser
• A catalogue of papers about AMR parsing

There is a very good fairly deep NLP-and-commonsense series of podcasts and other interesting stuff from the Allen Institute for AI:

• NLP highlights podcasts
• episode 89 about dialog systems
• episode 91 about executable semantic parsing
• episode 59 about semantic parsing
• demos, in partcular Rover demo

Lecture: further details for semantic parsing: 26 April

Teams recording

Reasoning with uncertainty: discrete reasoning options

Lecture recording in teams.

This week is devoted to discrete methods. Please read the materials described below slowly and carefully: the E1 materials are what I expect you to know well at the exam and the E2 materials are what you should know a bit at the exam.

Experimentation with the DLV system is recommended!

Roughly said, there are three kinds of discrete methods:

• Exceptions to rules. The main method is called default logic. E1 Read carefully!. Another, less popular method is circumscription (skim through, careful reading not needed). The main classical example for default logic is "birds normally fly": bird(X) & [we fail to prove -flies(X)] => flies(X). In other words, a rule is blocked, if we manage to prove that a negation of the rule-derived fact can be proved. Say we have
  • bird(tweety)
  • penguin(pengu)
  • (bird(X) & [we fail to prove -flies(X)]) => flies(X)
  • penguin(X) => -flies(X)

Then we can derive -flies(pengu) and flies(tweety) but we cannot derive flies(pengu), since it is blocked by -flies(pengu) which we could derive. In the wikipedia page above the "justification P" really means "we fail to prove -P".

Esimest järku loogikas ei ole üldjuhul võimalik välja rehkendada, et mingi asi ei ole järelduv. Lausearvutuses samas on. Sestap on olemas selline tore valdkond nagu answer set programming, E2 kus esimest järku muutujatega valem teisendatakse lausearvutuseks, tekitades muutujatega reeglitest hästi palju konstante sisaldavaid reegleid. Siis muutuvad blokkerite arvutamised võimalikuks. Võid tahta lugeda päris head introt teemasse, mille on kirjutanud Aalto ülikooli rektor Niemelä. Samas ei ole see lausearvutuseks-teisendamine mittetriviaalsetel juhtudel võimalik. Üks paremaid süsteeme seal on DLV (DLV wikis), vaata DLV manuaali Kui sul tekib rohkem huvi, siis selle answer-set programming lähenemise jaoks on olemas terve kursus videote ja loengumaterjalidega.

Teine oluline küsimus eranditega reeglite puhul on nende prioriteedid: kui kaks reeglit üksteist blokivad, kas mõnel peaks olema kõrgem prioriteet? Näiteks, "materiaalsed asjad üldiselt ei lenda", "linnud on materiaalsed asjad", "linnud üldiselt lendavad", "pingiviinid on linnud", "pingviinid üldiselt ei lenda". Siin paistab, et taksonoomias allpool olevaid asju kirjeldavad reeglid võiks olla kõrgema prioriteediga kui ülalpool asju kirjeldavad reeglid. Samas "Nixoni kolmnurga" näites "vabariiklased ei ole patsifistid", "kveekerid on patsifistid", "Nixon korraga vabariiklane ja kveeker" oleks loomulik mõelda, et esimesed kaks default reeglit Nixoni jaoks blokeeruvadki ja me ei saa järelda, et Nixon on patsifist, ega ka seda, et ta ei ole patsifist. Prioriteetide teema kohta võib vaadata täeindavalt seda artiklit või seda sarnast artiklit.

• Describing beliefs of various actors. This builds upon modal logic (see intro presentation E2 to understand the basics) and the main method is (auto)epistemic logic: the logic for encoding who knows or believes what. See the long tutorial as an easily read text E2 and additionally tutorial slides Ijcai93.pdf by the same authors to accompany the long text. Read carefully, starting with the long text and also looking at slides at the side.
• Multi-valued logics with belief degrees (a la "very unlikely", "very likely") at different intervals. We will not look into these: we'll investigate the numeric methods instead. Have a very brief look at the wikipedia.

NB! None of these methods are really used in any kinds of practical systems: they are hard to efficiently implement and they are not understood well enough in practical settings.

Reasoning with uncertainty: combining discrete and numeric

Read Implementing_Default_Logic.pdf draftE2.

We will look at a number of small examples using both numeric and discrete uncertainty and will experiment with our new experimental solver in a live session.

• What is our approach to using default rules?
• How do we handle numeric uncertainty and default rules together?
• What works OK and what is problematic?
• What are some important practical issues to be solved?
• Some ideas for building a knowledge base with uncertainties.

See also draft notes for our approach to reasoning with defaults

Context and relevance

Märkmed konteksti teemal

Consultation for exam

Other stuff: Commonsense rules development