tags : sqlite, SQL, Data Visualization, Data Engineering, Geography
FAQ
SQL injection?
- Picked from Arc Note: Datasette - Simon Willison
- DB opened with readonly
- Has filter on query for safe statements
The ecosystem
Datasette is built on top of sqlite, a tool for exploring and publishing data. Eg. You can create a website/API out of your data. It can be combined with other tools and extended with plugins.
Distributions
These are different ways datasette is distributed, in a way these also fall under what we consider “tools” under datasette ecosystem.
- Datasette: Python web application, CLI, Server, (can be deployed)
- Datasette Lite : In the browser (Runs datasette in pyodide in a web worker, see WebAssembly). Uses datasette-client.
- “So each time the user clicks a link in the parent window, that URL is sent to the web worker where Datasette generates the full HTML of the page just as if it was running as a regular server-side application.”
- “The web worker sends that HTML (and the status code and content-type) back to the parent window, which injects it into the page using innerHTML.”
- Datasette Desktop: macOS desktop application (Electron), uses datasette-app-support and python-build-standalone
Official Tools(46)
These can be run independently but also has useful usecases when used with datasette. See Datasette Tools.
<source>-to-sqlite: The primary idea is that, we somehow dump the data into sqlite and then we can easily explore the data using datasette. Source can be anything. There are ~20 such official tools. You can ofcourse build your own.other tools: These are tools that usually can be considered utilities, some of these are related to datasette, some are related to sqlite, some may not be related to any but indirectly can be useful etc. Following are some picked tools that I think are useful to me currently.- sqlite-utils
- shot-scraper (mentioned in Archival)
- strip-tags
- db-to-sqlite
- git-history
- airtable-export
- markdown-to-sqlite
- geo (See Geography)
- shapefile-to-sqlite
- geocode-sqlite
- geojson-to-sqlite
- download-tiles
Official Plugins(136)
Plugins extend Datasette with new functionality, Data Visualization, Authentication, more data processing, other post/pre processing/debugging stuff. You can build your own. Following are some picked plugins that I think are useful to me currently. Some(sqlite-X) of these plugins directly work as sqlite extensions aswell, asg017 maintains a log of these. Datasette uses pluggy for this.
See https://datasette.io/content/plugins?_facet=tags
- Query Enhancements
- datasette-saved-queries
- datasette-query-history
- datasette-template-sql
- datasette-search-all
- datasette-leaflet-freedraw: query a database by drawing shapes on a map
- datasette-write / datasette-write-ui
- Render enhance
- datasette-hovercards
- datasette-dashboards
- datasette-render-timestamps
- datasette-multiline-links (🤌)
- datasette-render-image-tags
- datasette-copyable
- datasette-leaflet-geojson
- datasette-geojson-map
- Depends on datasette-geojson
- Eg. useful when we use something like
geojson-to-sqlite, and the column in binary now
- Other Enhancements
- datasette-scraper (See Scraping)
- datasette-comments
- datasette-enrichments: See plugin framework for augmenting your data, enrichments themselves are plugins too.
- sqlite-chronicle: similar to sqlite-history but instead creates a separate chronicle table to keep track and external applications can make use of it.
- Interop
- datasette-export-notebook : exporting data to Jupyter or Observable
- Debugging
- datasette-show-errors
- datasette-edit-templates
- datasette-psutil
- datasette-pyinstrument
- datasette-total-page-time
- Security/Performance/Infra
- datasette-cors
- datasette-init
- datasette-rure / sqlite-regex
- datasette-gzip
- datasette-backup
- datasette-hashed-urls
- datasette-sqlite-fts4
- datasette-publish-fly
- datasette-block-robots
- datasette-configure-fts
- datasette-scale-to-zero
- Geo
- datasette-tiles: serving MBTiles map tiles
- datasette-cluster-map
- datasette-leaflet
- datasette-basemap
- datasette-placekey
- datasette-haversine
- sqlite-colorbrewer
- sqlite-tg / datasette-sqlite-tg (as an alternative to spatialite at times)
- Viz
- datasette-vega : (demo seems to be facing CORS issues currently)
- datasette-nteract-data-explorer
Related/Spunoff projects
- Dogsheep : This is a separate project by the author of datasette which re-uses the datasette tools for personal analytics/self quantification etc.
- datasette-parquet: Even though datasette does not directly work with DuckDB, this allows you to do so.