Website
This loader collects raw text from websites, ignoring images and code blocks. It can refresh and track URLs similar to other loaders, identifying changes in the website structure and prompting the user accordingly. Future features include parsing media (images, videos, code blocks) and flagging data conflicts or unwanted elements like PII.
Client Configs: Base URL, Auto-Refresh Rate, Exclude URL List.
Additional Functionality: Refresh Now (overrides auto-refresh rate).
Last updated