covid19/keywords/README.md

# Keywords

This code finds trending web searches related to the COVID-19 pandemic using Google trends (`collect_trends.py`). It then searches for relevant keywords on Wikidata (`wikidata_search`) in order to find high-quality translations of important words and phrases (`wikidata_translations.py`). The goal is to support efforts expanding the Observatory to information in many languages beyond English.  

We search the Wikidata API for entities in `src/wikidata_search.py` and then we make simple SPARQL queries in `src/wikidata_translations.py` to collect labels and aliases the entities.  The labels come with language metadata.  This seems to provide a decent initial list of relevant terms across multiple languages. 

The output data lives at [covid19.communitydata.science](https://covid19.communitydata.science/datasets/keywords).

The output files have 4 colums: 

- `itemid` links to the wikidata entity
- `label` is the translation of the relevant keyword
- `langcode` is the [iso 639-1](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) code corresponding the language of the label. 
- `is_alt` indicates whether the label is an [alias](https://www.wikidata.org/wiki/Help:Aliases).
Improve README.md for keywords 2020-03-31 22:25:51 +00:00			`# Keywords`
Finish MVP for transliterations code is reasonably well-written checked that we get seemingly good data back adding README adding data 2020-03-25 05:06:08 +00:00
Improve README.md for keywords 2020-03-31 22:25:51 +00:00			This code finds trending web searches related to the COVID-19 pandemic using Google trends (`collect_trends.py`). It then searches for relevant keywords on Wikidata (`wikidata_search`) in order to find high-quality translations of important words and phrases (`wikidata_translations.py`). The goal is to support efforts expanding the Observatory to information in many languages beyond English.

			We search the Wikidata API for entities in `src/wikidata_search.py` and then we make simple SPARQL queries in `src/wikidata_translations.py` to collect labels and aliases the entities. The labels come with language metadata. This seems to provide a decent initial list of relevant terms across multiple languages.

			`The output data lives at [covid19.communitydata.science](https://covid19.communitydata.science/datasets/keywords).`
add documentation for the output files 2020-03-31 23:22:30 +00:00
			`The output files have 4 colums:`

			- `itemid` links to the wikidata entity
			- `label` is the translation of the relevant keyword
			- `langcode` is the [iso 639-1](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) code corresponding the language of the label.
			- `is_alt` indicates whether the label is an [alias](https://www.wikidata.org/wiki/Help:Aliases).