Andme-apide märkmed loengust: erinevus redaktsioonide vahel
| (ei näidata sama kasutaja 6 vahepealset redaktsiooni) | |||
| 1. rida: | 1. rida: | ||
== General == | <!-- == General == | ||
Check out [https://market.mashape.com/ market.mashape.com] | Check out [https://market.mashape.com/ market.mashape.com] | ||
--> | |||
<!-- | <!-- | ||
[http://www.programmableweb.com/ Programmable web] is a great site giving pointers to a large set of open data apis people have created. | [http://www.programmableweb.com/ Programmable web] is a great site giving pointers to a large set of open data apis people have created. | ||
--> | --> | ||
Use either or both: | <!-- Use either or both: | ||
* curl to try the examples out or use the small python code down below to ease experimenting | * curl to try the examples out or use the small python code down below to ease experimenting | ||
* https://www.getpostman.com/ app | * https://www.getpostman.com/ app --> | ||
<!-- | <!-- | ||
== Background example == | == Background example == | ||
| 41. rida: | 42. rida: | ||
== Foursquare == | == Foursquare == | ||
Example (replace MYTOKEN, you can make it on the docs site or by going to https://developer.foursquare.com/ | Example (replace MYTOKEN, you can make it on the docs site or by going to https://developer.foursquare.com/) | ||
https://api.foursquare.com/v2/venues/search?ll=59.437,24.745&radius=1000&&oauth_token=MYTOKEN&v=20141001&limit=50&intent=browse | https://api.foursquare.com/v2/venues/search?ll=59.437,24.745&radius=1000&&oauth_token=MYTOKEN&v=20141001&limit=50&intent=browse | ||
| 59. rida: | 60. rida: | ||
See [https://dev.twitter.com/docs twitter dev docs] and [https://dev.twitter.com/docs/using-search twitter search docs] | See [https://dev.twitter.com/docs twitter dev docs] and [https://dev.twitter.com/docs/using-search twitter search docs] | ||
<!-- | |||
API testing console: https://dev.twitter.com/rest/tools/console | API testing console: https://dev.twitter.com/rest/tools/console | ||
--> | |||
== Google search == | == Google search == | ||
| 69. rida: | 71. rida: | ||
See [https://developers.google.com/web-search/ web search docs] and [https://developers.google.com/custom-search/v1/overview custom-search docs] | See [https://developers.google.com/web-search/ web search docs] and [https://developers.google.com/custom-search/v1/overview custom-search docs] | ||
See also [http://stackoverflow.com/questions/4082966/what-are-the-alternatives-now-that-the-google-web-search-api-has-been-deprecated/22494400#22494400 old stackoverflow discussion] on alternatives and https://serpapi.com/ | |||
Pricing: | |||
* [https://blog.api.rakuten.net/top-10-best-search-apis/ overview of some major search apis] | |||
* [https://blog.expertrec.com/site-search-pricing-google-cse-replacement/ 3d party sales pitch overview of google search apis with features and prices] | |||
* [https://cloud.google.com/maps-platform/pricing/sheet maps pricing] | |||
* [https://developers.google.com/maps/documentation/places/web-service/usage-and-billing places api pricing] deprecated? parts do not work? | |||
== Facebook graph api == | == Facebook graph api == | ||
| 126. rida: | 134. rida: | ||
main2() | main2() | ||
</pre> | </pre> | ||
Viimane redaktsioon: 18. märts 2021, kell 09:14
Weather
EMHI example http://www.ilmateenistus.ee/ilma_andmed/xml/observations.php
Foursquare
Example (replace MYTOKEN, you can make it on the docs site or by going to https://developer.foursquare.com/)
Google places
Example (replace MYKEY, you can make it on the docs site) https://maps.googleapis.com/maps/api/place/search/json?location=-33.8670522,151.1957362&radius=500&types=food&name=harbour&sensor=false&key=MYKEY
Example url from old api, which is no longer functional: http://search.twitter.com/search.json?q=%40tallinn
See twitter dev docs and twitter search docs
Google search
Example (sign in to your Google account and read docs about how to create a custom search. Then create a new engine and put its id in place of MYENGINE below. Also replace MYKEY, you can make it on the docs site) https://www.googleapis.com/customsearch/v1?cx=MYENGINE&key=MYKEY&q=lectures
See web search docs and custom-search docs
See also old stackoverflow discussion on alternatives and https://serpapi.com/
Pricing:
- overview of some major search apis
- 3d party sales pitch overview of google search apis with features and prices
- maps pricing
- places api pricing deprecated? parts do not work?
Facebook graph api
https://developers.facebook.com/docs/graph-api
Quick example: https://graph.facebook.com/me?fields=id,name&access_token=MY_TOKEN
you can get MY_TOKEN from several places, for example: https://developers.facebook.com/tools/explorer?method=GET&path=me%3Ffields%3Did%2Cname&version=v2.1
Check also non-api scraping:
- http://www.pcworld.com/article/2056080/facebook-stalker-tool-uses-graph-search-for-powerful-data-mining.html
- https://github.com/milo2012/osintstalker
Python example for experimenting
#!/usr/bin/python from math import * import os import urllib2 import json import xml.dom.minidom # NB! Replace YOURKEY and YOURTOKEN below with a key and token you make yourself # on the corresponding api doc page #url="https://api.foursquare.com/v2/venues/search?ll=59.437,24.745&radius=1000&&oauth_token=YOURTOKEN&v=20120103&limit=50&intent=browse" #url="https://maps.googleapis.com/maps/api/place/search/json?location=-33.8670522,151.1957362&radius=500&sensor=false&key=YOURKEY" #url="https://www.googleapis.com/customsearch/v1?key=YOURKEY&q=lectures" def main(): f=urllib2.urlopen(url) s=f.read() data=json.loads(s) #print data print json.dumps(data, sort_keys=True, indent=2) def main2(): req=urllib2.Request(url=url) rawresp=urllib2.urlopen(req) print "headers:\n" h=rawresp.info().headers for e in h: print e, s=rawresp.read() data=json.loads(s) print "\ndata:\n" print json.dumps(data, sort_keys=True, indent=2) main2()