How to fetch a web page as Markdown

This guide shows how to let the model read a web page, or save it to a file, with the built-in fetch_content tool. It assumes you already run mm with a configuration file.

Steps

Turn the tool on in your configuration file (it is off by default):
```
fetchContentTool: true
```
Start mm and check that the start-up banner lists fetch_content among the tools.
Ask for the page in plain words, in any language:
```
fetch the content of https://k33g.org
```
```
récupère le contenu de https://k33g.org
```
The terminal shows 🌐 fetch_content: https://k33g.org, then the first lines of the Markdown, and the model answers from it.

Save the page to a file instead of showing it

Name the file and say not to display it:

récupère le contenu de https://hello.com mais ne l'affiche pas et sauve le dans hello.txt

The model passes output: hello.txt. The Markdown is written to hello.txt (relative to the directory mm runs in, or to the project opened in the editor) and the tool returns only one line:

[fetch_content: saved 9 lines of Markdown (421 bytes) from https://hello.com to hello.txt — the content is not shown]

The page does not enter the conversation, so a long page does not use up the context window.

Point the agent at a list of pages

To make the agent read given pages first, list them in the system prompt. No other setting is needed:

fetchContentTool: true
system: |
  You answer questions about mini-me from its documentation.
  Before answering, call fetch_content on the page of this list that fits the
  question, and answer from it. Do not use any other URL.

  - How to add a skill: https://rickub.com/bots-garden/mini-me/blob/main/docs/en/how-to/add-a-skill.md
  - How to keep sessions: https://rickub.com/bots-garden/mini-me/blob/main/docs/en/how-to/keep-sessions.md

Give each URL a few words that say when to use it: this is what the model chooses from.

This list guides the model but does not restrict it. Asked for another site, the model can still fetch it, even with Do not use any other URL. To block URLs, use a hook: see guard the agent with hooks.

The list is sent with every request. For more than a few pages, consider a skill: see add a skill.

Variants

A URL typed carelessly. k33g.org and https//k33g.org both work: a missing scheme becomes https://, a missing colon is added back.
Complex tables. Tables with merged cells are kept as raw HTML by default. Ask for flattened tables ("…as Markdown tables, flatten them") and the model passes tables: flatten.
Strikethrough and task lists. Ask for them explicitly ("keep the struck-through text", "keep the checkboxes") and the model passes strikethrough: true or taskList: true.
Links relative to another site. If the page is a mirror, ask to resolve links against the original; the model passes baseUrl.
With yolo: false. Reading a page needs no permission; saving it to a file asks first, like write_file.

When it fails

The model receives [fetch_content: error N (…) — reason] and usually tells you why:

error 2 (input): the site is unreachable, returned an HTTP error, or its robots.txt forbids robots to read that page. mm does not bypass robots.txt.
error 1 (conversion): the URL is not an HTML page (a PDF, JSON, an image) or is larger than 5 MiB.
error 3 (usage): the call itself is wrong, for example an ftp:// URL.

If the model uses curl through bash instead, the tool is probably not declared: check fetchContentTool: true and the banner.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86


# How to fetch a web page as Markdown

This guide shows how to let the model read a web page, or save it to a file, with the built-in `fetch_content` tool. It assumes you already run `mm` with a configuration file.

## Steps

1. Turn the tool on in your configuration file (it is off by default):

   ```yaml
   fetchContentTool: true
   ```

2. Start `mm` and check that the start-up banner lists `fetch_content` among the tools.

3. Ask for the page in plain words, in any language:

   ```text
   fetch the content of https://k33g.org
   ```

   ```text
   récupère le contenu de https://k33g.org
   ```

   The terminal shows `🌐 fetch_content: https://k33g.org`, then the first lines of the Markdown, and the model answers from it.

## Save the page to a file instead of showing it

Name the file and say not to display it:

```text
récupère le contenu de https://hello.com mais ne l'affiche pas et sauve le dans hello.txt
```

The model passes `output: hello.txt`. The Markdown is written to `hello.txt` (relative to the directory `mm` runs in, or to the project opened in the editor) and the tool returns only one line:

```text
[fetch_content: saved 9 lines of Markdown (421 bytes) from https://hello.com to hello.txt — the content is not shown]
```

The page does not enter the conversation, so a long page does not use up the context window.

## Point the agent at a list of pages

To make the agent read given pages first, list them in the system prompt. No other setting is needed:

```yaml
fetchContentTool: true
system: |
  You answer questions about mini-me from its documentation.
  Before answering, call fetch_content on the page of this list that fits the
  question, and answer from it. Do not use any other URL.

  - How to add a skill: https://rickub.com/bots-garden/mini-me/blob/main/docs/en/how-to/add-a-skill.md
  - How to keep sessions: https://rickub.com/bots-garden/mini-me/blob/main/docs/en/how-to/keep-sessions.md
```

Give each URL a few words that say when to use it: this is what the model chooses from.

This list guides the model but does not restrict it. Asked for another site, the model can still fetch it, even with `Do not use any other URL`. To block URLs, use a hook: see [guard the agent with hooks](guard-the-agent-with-hooks.md).

The list is sent with every request. For more than a few pages, consider a skill: see [add a skill](add-a-skill.md).

## Variants

- **A URL typed carelessly.** `k33g.org` and `https//k33g.org` both work: a missing scheme becomes `https://`, a missing colon is added back.
- **Complex tables.** Tables with merged cells are kept as raw HTML by default. Ask for flattened tables ("…as Markdown tables, flatten them") and the model passes `tables: flatten`.
- **Strikethrough and task lists.** Ask for them explicitly ("keep the struck-through text", "keep the checkboxes") and the model passes `strikethrough: true` or `taskList: true`.
- **Links relative to another site.** If the page is a mirror, ask to resolve links against the original; the model passes `baseUrl`.
- **With `yolo: false`.** Reading a page needs no permission; saving it to a file asks first, like `write_file`.

## When it fails

The model receives `[fetch_content: error N (…) — reason]` and usually tells you why:

- `error 2 (input)`: the site is unreachable, returned an HTTP error, or its `robots.txt` forbids robots to read that page. `mm` does not bypass `robots.txt`.
- `error 1 (conversion)`: the URL is not an HTML page (a PDF, JSON, an image) or is larger than 5 MiB.
- `error 3 (usage)`: the call itself is wrong, for example an `ftp://` URL.

If the model uses `curl` through `bash` instead, the tool is probably not declared: check `fetchContentTool: true` and the banner.

## See also

- Every field, rule and error code: [built-in tools reference](../reference/tools.md#fetch_content)
- The configuration key: [configuration reference](../reference/configuration.md)
- Why the tool works this way: [the `fetch_content` tool](../explanation/fetch-content-tool.md)