mirror of
https://gitverse.ru/kpa39l/openspec-lab.git
synced 2026-09-29 09:15:01 +00:00
OpenSpec: исследование + tavily-proxy-setup, local-extractor (spec-driven для инфраструктуры)
This commit is contained in:
@@ -0,0 +1,34 @@
|
||||
# Delta for web-extract-local
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Local Extraction Mode
|
||||
The system MUST support extracting page content fully locally (no cloud API) from the same HTTP endpoint used by the Tavily provider.
|
||||
|
||||
#### Scenario: Extract page locally
|
||||
- GIVEN tavily_extract_proxy.py runs with --local (or --local-socks)
|
||||
- WHEN a POST /extract with {"urls":["https://..."]} is sent
|
||||
- THEN the server fetches each URL, converts to markdown with trafilatura
|
||||
- AND returns Tavily-shaped JSON: {"results":[{"url","title","raw_content","metadata"}], "failed_results":[], "failed_urls":[]}
|
||||
|
||||
#### Scenario: Page fails to fetch
|
||||
- GIVEN a URL is unreachable or returns HTTP >= 400
|
||||
- WHEN extraction is attempted
|
||||
- THEN the URL lands in failed_results with an error message
|
||||
- AND other URLs still process normally
|
||||
|
||||
### Requirement: SOCKS Support for Local Fetches
|
||||
The local extractor MUST be able to fetch through the SOCKS5 tunnel (--local-socks 127.0.0.1:1080) for sites blocked from RU IPs.
|
||||
|
||||
#### Scenario: Fetch through tunnel
|
||||
- GIVEN --local-socks 127.0.0.1:1080 is set
|
||||
- WHEN page fetch happens
|
||||
- THEN the request is routed through the SOCKS5 tunnel
|
||||
|
||||
### Requirement: Backward Compatible Interface
|
||||
The local mode MUST NOT break the existing Tavily forward mode: the same port/endpoint keeps working for both.
|
||||
|
||||
#### Scenario: Forward still works
|
||||
- GIVEN the service runs with --socks (forward mode)
|
||||
- WHEN POST /extract is sent
|
||||
- THEN the request is forwarded to api.tavily.com as before
|
||||
Reference in New Issue
Block a user