dotnet tool install --global dotnet-soddi --version 0.3.2
dotnet new tool-manifest # if you are setting up this repo dotnet tool install --local dotnet-soddi --version 0.3.2
nuke :add-package dotnet-soddi --version 0.3.2
DotNet-Soddi (Stack Overflow Data Dump Importer)
dotnet tool install --global dotnet-soddi --version 0.2.1
Inspirited by the original Soddi, DotNet-Soddi is a console application that assists in not just importing the Stack Overflow data dumps, but also obtaining them.
- Discover archives from the command line.
- Importing data from xml files or stream directly from the .7z archive files.
- Download archive files from the command line via HTTP based on their site name.
- Download archive files via BitTorrent based on their site name.
Not all features of Soddi are supported.
- It's only for SQL Server.
- Full Text isn't supported.
Download and install a database in two lines
The following command will download the archive for the math.stackexchange.com, then import it into an existing database named math.stackexchange.com
soddi download math soddi import math -d math.stackexchange.com
Because of the size of the database and the bandwidth of archive.org, you might be better off using the torrent option. The following command will do everything required to connect to the appropriate trackers and peers to download the math database and exit upon completion.
soddi torrent math
Downloads a Stack Overflow site archive via HTTP from archive.org.
USAGE: download archive for aviation.stackexchange.com: soddi download aviation download archive for math.stackexchange.com to a particular folder: soddi download --output c:\stack-data math pick from archives containing stack and download: soddi download --pick stack -o, --output Output folder -p, --pick (Default: false) Pick from a list of archives to download --help Display this help screen. --version Display version information. Archive (pos. 0) Required. Archive to download
List available Stack Overflow data dumps.
USAGE: List all archives: soddi list List all archives containing the letters "av": soddi list av List all archives containing the letters "av" including meta sites: soddi list --includeMeta av --includeMeta (Default: false) Include meta databases. --help Display this help screen. --version Display version information. Pattern (pos. 0) (Default: ) Pattern to include (e.g. "av" includes all archives containing "av").
Import data from a 7z archive or folder
USAGE: Import data using defaults: soddi import math.stackexchange.com.7z Import data from a folder containing a collection of .7z or .xml files: soddi import stackoverflow Import data using a connection string and database name: soddi import --connectionString "Server=(local)\Sql2017;User Id=admin;password=t3ddy" --database math math.stackexchange.com.7z Import data using defaults and create database: soddi import --dropAndCreate math.stackexchange.com.7z Import data using defaults without constraints: soddi import --skipConstraints math.stackexchange.com.7z -d, --database Name of database. If omitted the name of the file or folder will be used. -c, --connectionString (Default: Server=.;Integrated Security=true) Connection string to server. Initial catalog will be ignored and the database parameter will be used for the database name. --dropAndCreate (Default: false) Drop and recreate database. If a database already exists with this name it will be dropped. Then a database will be created in the default server file location with the default server options. --skipConstraints (Default: false) Skip adding primary keys and unique constraints. --skipTags (Default: false) Skip adding PostTags table. --help Display this help screen. --version Display version information. Path (pos. 0) Required. File or folder containing xml archives. The file must be a .7z file. If using a folder it can contain either .7z or .xml content
Download database via BitTorrent. For larger databases this might prove significantly faster.
USAGE: Download files associated with the math site from the torrent file: soddi torrent math Download to a specific folder: soddi torrent --output "c:\torrent files" math Enable port forwarding: soddi torrent --portForward math Pick from archives containing "stack": soddi torrent --pick stack -o, --output Output folder -f, --portForward (Default: false) Experimental. Enable port forwarding -p, --pick (Default: false) Pick from a list of archives to download --help Display this help screen. --version Display version information. Archive (pos. 0) Required. Archive to download
Super experimental. Downloads and extracts one of the BrentO provided databases via BitTorrent.
If you don't supply an
ARCHIVE_NAME you will be prompted for one.
USAGE: soddi brent [ARCHIVE_NAME] [OPTIONS] EXAMPLES: soddi brent soddi brent small soddi brent medium soddi brent large soddi brent extra-large ARGUMENTS: [ARCHIVE_NAME] Archive to download OPTIONS: -h, --help Prints help information -o, --output Output folder --skipExtraction Don't extract the downloaded 7z files -f, --portForward Experimental. Enable port forwarding
|Product||Versions Compatible and additional computed target framework versions.|
|.NET||net7.0 is compatible. net7.0-android was computed. net7.0-ios was computed. net7.0-maccatalyst was computed. net7.0-macos was computed. net7.0-tvos was computed. net7.0-windows was computed. net8.0 was computed. net8.0-android was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed.|
This package has no dependencies.