Methabot
From Seo Wiki - Search Engine Optimization and Programming Languages
Jump to navigationJump to search
| This article may not meet the general notability guideline. Please help to establish notability by adding reliable, secondary sources about the topic. If notability cannot be established, the article is likely to be merged, redirected, or deleted. (March 2009) |
| Developer(s) | Emil Romanus |
|---|---|
| Stable release | 1.7.0 / June 23, 2009 |
| Written in | C |
| Operating system | FreeBSD, Linux |
| Type | web crawler open source |
| License | ISC licence |
| Website | http://metha-sys.org/ |
Methabot is a scriptable web crawler designed for flexibility and speed. It is free software written in C, distributed under the terms of the ISC licence.
Methabot has wide support for customization. It can be scripted using Javascript with E4X, configured using its own configuration language, and dynamically switch configuration while running.
Key features
- Scriptable using Javascript
- Provides MySQL bindings to Javascript
- Support for the Robots Exclusion Standard
- User-defined filetype filtering and sorting, according to custom rules
- Heavy multi-threading
- Chaining of custom parsers
- Converts HTML to real XML for E4X compatibility
External links
| Stub icon | This free software-related article is a stub. You can help Wikipedia by expanding it. |
If you like SEOmastering Site, you can support it by - BTC: bc1qppjcl3c2cyjazy6lepmrv3fh6ke9mxs7zpfky0 , TRC20 and more...
→
Categories:
- Pages where expansion depth is exceeded
- Pages with broken file links
- Articles with topics of unclear notability from March 2009
- Articles with invalid date parameter in template
- All articles with topics of unclear notability
- Internet search engines
- Free search engine software
- Free web crawlers
- Free software stubs
