Build an AI Research Workflow with Search, Extraction, and Citations

TutorialsSeptember 7, 2026·AIsa

Build a Python AI research workflow with AIsa: search the web, extract selected pages, validate citation references, and handle missing evidence.

AI agent research workflowAIsa APITavily SearchTavily ExtractcitationsPythonAI agents

An AI research answer needs more than a list of links at the bottom. It needs a connection between each conclusion, the source that supports it, and the text the application actually retrieved.

Consider a research assistant comparing developer tools. Search may find a product page, an outdated comparison, and a discussion describing a problem that has since been fixed. Passing all three snippets to a model and asking for citations does not resolve those differences. A real URL can still accompany an unsupported conclusion.

This tutorial builds a small Python workflow that keeps discovery, reading, and synthesis separate. It searches for candidate pages, extracts selected content, generates a structured answer, and checks source references before rendering Markdown citations.

The example uses AIsa's Tavily Search and Tavily Extract endpoints, followed by a text model through the AIsa model gateway. It is a single-provider retrieval example, not a benchmark or a claim that one search covers the whole web.

Separate Search Results from Supporting Evidence

There are three different objects in this workflow:

ObjectWhat it containsWhat it establishes
Search resultA title, URL, and search excerptA potentially relevant page was discovered
Retrieved sourceA URL and the page content returned by extractionSome text from that page was available to the application
Cited statementAn answer statement linked to a source and supporting excerptA specific source is being offered for that statement

A search result is not automatically a citation. A successful extraction does not establish that a page is correct. A citation is not proof that the source supports the model's interpretation.

Keeping these objects separate makes failures easier to locate. If search found the right page but extraction failed, that is a retrieval problem. If the text was available but the answer misrepresented it, that is a synthesis problem.

Figure 1 shows the five stages used in the example. The main path carries candidate URLs into extraction, then supplies a source map to the model. The red branches stop an empty retrieval or reject an invalid draft. When only some extractions fail, the workflow can continue with the available text while disclosing the missing coverage.

Five-stage research workflow: search and select URLs, extract and label sources, generate a draft, validate references, and render citations. Empty retrieval stops the run; invalid drafts are rejected.

Figure 1. Search, extraction, synthesis, reference checks, and presentation remain separate. Counts shown are tutorial limits, not service limits. Passing reference checks does not establish factual correctness.

Build the Retrieval Step

Use Python 3.10 or later. The example uses only the standard library. Configure AISA_API_KEY as a server-side environment variable, and set AISA_MODEL to an enabled text model that supports /v1/chat/completions. Available model IDs and routes are listed in the model catalog; the example deliberately does not assume a particular model is enabled for every account.

Search and extraction use https://api.aisa.one/apis/v1. Model generation uses https://api.aisa.one/v1. These are different URL prefixes.

The first part searches for up to five results and selects at most three distinct URLs for extraction:

python

The source limit and 8,000-character cap are tutorial controls, not API limits or quality thresholds. Selecting the first three results is also a simple starting policy, not an authority assessment. A more demanding workflow should select pages for relevance, primary-source status, freshness, and diversity.

This example deduplicates exact URLs after removing fragments. It does not claim to resolve canonical URLs or detect syndicated content. It also skips extraction results whose returned URL differs from the selected URL; production redirect handling should retain and verify that relationship rather than guessing it.

Search can return useful excerpts, and its include_raw_content option can request additional content. A separate extraction call is useful when the application needs a clearer reading step for selected URLs. If usable page content is already present, avoid fetching it again unnecessarily. See the Search request and response fields.

Extraction can succeed overall while individual pages fail. The Extract response exposes failed_results, so checking only the HTTP status is not enough. The example reports those failures and stops if no usable text remains.

Generate Statements, Not Model-Invented URLs

Assign source IDs before generation. The model receives the question and a source map, then returns statements referencing those IDs. The application, not the model, constructs the final source links.

The second code block continues the same script:

python

This uses a JSON instruction rather than assuming all routed models implement the same structured-output feature. The next step rejects malformed output. Where a selected model supports schema-constrained output, that can reduce formatting failures, but it does not establish factual correctness.

The supporting quotes are part of the draft's evidence record. They make review more specific than checking whether the answer contains clickable links.

Validate and Render the Citations

Before displaying the answer, check that every statement references known sources and that each quoted excerpt occurs in the exact text supplied to the model:

Figure 2 follows one illustrative source, S1, into a proposed statement. The checks confirm that S1 exists and that the attached quote appears in its retrieved text. An unknown ID or invented quote fails. Whether the quote actually supports the statement remains a separate semantic question.

An illustrative source S1 and its excerpt connect to a proposed statement. Known source IDs and exact quote matches pass reference checks; unknown IDs and invented quotes are rejected. Semantic support, accuracy, and completeness remain unverified.

Figure 2. Source-reference and quote validation. All example text is illustrative, not a live API result. The source map supplies the URL; source correctness and claim support require additional review.

The final code block implements those reference checks:

python

Run the three Python blocks together in order. Search results, source counts, and generated statements will vary; there is no fixed answer that a successful run must reproduce.

These checks establish that a reference points to a supplied source and that the quote exists in the supplied excerpt. They do not prove that the quote supports the statement, that the page is accurate, or that important contrary evidence was found. Those require semantic review and, where appropriate, independent sources. For publication or high-stakes decisions, review the statement, quote, and surrounding source text together.

For an application UI, sanitize generated text before rendering Markdown or HTML. Keep the structured draft alongside the source map so a reviewer can inspect more than the rendered answer.

Add More Sources Only When the Question Needs Them

A multi-source search API is useful when a question needs different kinds of evidence, not simply more URLs. AIsa's search skills provide additional retrieval options, and the existing skills repository contains clients and examples.

Keep the roles distinct:

Research needSource type to considerInterpretation limit
An API's supported parametersOfficial documentation and release notesDocumentation may describe a different version
How a technique was evaluatedResearch papers and their methodsA result may not generalize beyond its setup
What users find difficultPublic discussions and issue reportsIndividual experiences are not population estimates
How a workflow looks in practiceTutorials and demonstrationsA demonstration is not a comparative benchmark

Independent search branches can run concurrently. Deduplicate their results, preserve which branch returned each source, and then select pages for reading. A repeated URL is one page discovered through multiple routes, not several independent pieces of evidence.

The minimal script above implements web search only. Adding social, academic, or video discovery requires those additional calls and their own response handling. A video title is not a transcript, and a social post should not become a verified product fact just because it appears beside official documentation.

For a visual implementation, the Dify multi-source search tutorial covers tool configuration and workflow assembly. The source-ID and validation pattern here can sit between retrieval and final answer rendering in either a visual workflow or a code-based agent.

Handle Failures Before Calling the Workflow Finished

FailureAppropriate response
Search finds no usable pagesReport insufficient evidence; revise the query deliberately
Some extractions failKeep the failure record and disclose reduced coverage
Every extraction failsStop; do not silently switch to an uncited answer
Authentication or access failsCheck credentials and permissions rather than repeatedly retrying
A timeout or rate limit occursApply bounded retries and backoff; respect retry guidance and possible duplicate cost
Model output is invalid or cites an unknown sourceReject the draft; a bounded regeneration can be a separate application policy
A quote exists but does not support the statementRevise or remove the statement after semantic review

The tutorial performs no automatic retries, caching, or persistent logging. Before scheduling repeated research, add those policies explicitly. Retain the question, queries, selected URLs, extraction failures, retrieval times, model ID, structured draft, and validation result. Keep API keys out of logs and model inputs. For user-supplied URLs, add public-address and redirect validation before enabling extraction.

The practical goal is a research answer whose evidence can be inspected. Search discovers candidates, extraction supplies readable text, and synthesis proposes conclusions. Source IDs and validation connect those stages without pretending that a citation alone makes a conclusion true.