RafaelEstevam.Simple.Spider.SqliteStorage
0.8.5
Sqlite-based storage engine to the SimpleSpider
See examples and documentation on the GitHub page
Install-Package RafaelEstevam.Simple.Spider.SqliteStorage -Version 0.8.5
dotnet add package RafaelEstevam.Simple.Spider.SqliteStorage --version 0.8.5
<PackageReference Include="RafaelEstevam.Simple.Spider.SqliteStorage" Version="0.8.5" />
For projects that support PackageReference, copy this XML node into the project file to reference the package.
paket add RafaelEstevam.Simple.Spider.SqliteStorage --version 0.8.5
The NuGet Team does not provide support for this client. Please contact its maintainers for support.
Simple.Spider
A simple and modular web spider written in C# .Net
- Core 3.1
- Net 5.0
This Package was moved from [Net.RafaelEstevam.Spider.Simple.Lib]
Content
Some advantages
- Very simple to use and operate, ideal for lots of small projects or personal ones
- Easy html filter with HObject (a HtmlNode wrap with use similar to JObject)
- Internal conversion from html to XElement, no need to external tools on use
- Automatic Json parser to JObject
- Automatic Json deserialize <T>
- Modular Parser engine (you can add your own parsers!)
- JSON and XML already included
- Modular Caching engine (you can add your own!)
- Stand alone Cache engine included, no need to external softwares
- Modular Downloader engine (you can add your own!)
- WebClient with cookies or HttpClient download engine included
Easy import with NuGet
Getting started
- Start a new console project and add Nuget Reference
- PM> Install-Package Net.RafaelEstevam.Spider.Simple.Lib
- Create a class for your spider (or leave in program)
- create a new instance of SimpleSpider
- Give it a name, cache and log will be saved with that name
- Give it a domain (your spider will not fleet from it)
- Add a event
FetchCompletedto - Optionally give a first page with
AddPage. If omitted, it will use the home page of the domain - Call
Execute()
void run()
{
var spider = new SimpleSpider("QuotesToScrape", new Uri("http://quotes.toscrape.com/"));
// Set the completed event to implement your stuff
spider.FetchCompleted += fetchCompleted_items;
// execute
spider.Execute();
}
void fetchCompleted_items(object Sender, FetchCompleteEventArgs args)
{
// walk around ...
// TIP: inspect args to see stuff
var hObj = args.GetHObject();
string[] quotes = hObj["span > .text"];
}
TIP: Use the Simple.Tests project to see examples and poke around
Samples
See all samples at Simple.Tests
Simple.Spider
A simple and modular web spider written in C# .Net
- Core 3.1
- Net 5.0
This Package was moved from [Net.RafaelEstevam.Spider.Simple.Lib]
Content
Some advantages
- Very simple to use and operate, ideal for lots of small projects or personal ones
- Easy html filter with HObject (a HtmlNode wrap with use similar to JObject)
- Internal conversion from html to XElement, no need to external tools on use
- Automatic Json parser to JObject
- Automatic Json deserialize <T>
- Modular Parser engine (you can add your own parsers!)
- JSON and XML already included
- Modular Caching engine (you can add your own!)
- Stand alone Cache engine included, no need to external softwares
- Modular Downloader engine (you can add your own!)
- WebClient with cookies or HttpClient download engine included
Easy import with NuGet
Getting started
- Start a new console project and add Nuget Reference
- PM> Install-Package Net.RafaelEstevam.Spider.Simple.Lib
- Create a class for your spider (or leave in program)
- create a new instance of SimpleSpider
- Give it a name, cache and log will be saved with that name
- Give it a domain (your spider will not fleet from it)
- Add a event
FetchCompletedto - Optionally give a first page with
AddPage. If omitted, it will use the home page of the domain - Call
Execute()
void run()
{
var spider = new SimpleSpider("QuotesToScrape", new Uri("http://quotes.toscrape.com/"));
// Set the completed event to implement your stuff
spider.FetchCompleted += fetchCompleted_items;
// execute
spider.Execute();
}
void fetchCompleted_items(object Sender, FetchCompleteEventArgs args)
{
// walk around ...
// TIP: inspect args to see stuff
var hObj = args.GetHObject();
string[] quotes = hObj["span > .text"];
}
TIP: Use the Simple.Tests project to see examples and poke around
Samples
See all samples at Simple.Tests
Release Notes
See examples and documentation on the GitHub page
Commit dd4d7ba
Dependencies
-
.NETStandard 2.1
- RafaelEstevam.Simple.Spider (>= 0.7.501)
- Simple.Sqlite (>= 0.4.79)
- System.Data.SQLite.Core (>= 1.0.113.7)
-
net5.0
- RafaelEstevam.Simple.Spider (>= 0.7.501)
- Simple.Sqlite (>= 0.4.79)
- System.Data.SQLite.Core (>= 1.0.113.7)
Used By
NuGet packages
This package is not used by any NuGet packages.
GitHub repositories
This package is not used by any popular GitHub repositories.
