Step 1: Construct and Fetch the First Page

To retrieve initial search results from FOFA’s free interface, encode the query string in Base64 and issue an HTTP GET request. The example below searches for assets matching the app="thinkphp" fingerprint.

import requests
import base64

query = 'app="thinkphp"'
base_url = 'https://fofa.so/result?qbase64='
encoded_query = base64.b64encode(query.encode('utf-8')).decode('utf-8')
full_url = base_url + encoded_query

response = requests.get(full_url)
if response.status_code == 200:
    html_content = response.content
    print(html_content.decode('utf-8'))
else:
    print(f'Failed to fetch page: {response.status_code}')

Step 2: Parse and Extract IP-Host URLs from HTML

Use lxml.etree to parse the returned HTML and extract target URLs from anchor tags within .re-domain containers.

from lxml import etree

# Assuming `html_content` is available from previous step
tree = etree.HTML(html_content)
urls = tree.xpath('//div[@class="re-domain"]/a[@target="_blank"]/@href')

# Join into newline-separated string
formatted_urls = '\n'.join(urls)
print(formatted_urls)

# Append to file
with open('targets.txt', 'a', encoding='utf-8') as f:
    f.write(formatted_urls + '\n')

Step 3: Paginate with Authentication Headers

FOFA’s free tier requires valid session cookies to access mutliple pages. Below is a loop that iterates through five result pages using a custom header dictionary containing _fofapro_ars_session and other required cookies.

import time

cookies = {
    '_fofapro_ars_session': '1e1e66e681f5ca085635005',
    'Hm_lvt_9490413c5eebdadf757c2be2c816aedf': '1616029,161698533,16447,16171152',
    'search_history': 'app%3D%22thinkphp%22',
    'referer_url': '%2Fresult%3Fq%3Dapp%253D%2522thinkphp%2522%26qbase64%3DYXBwPSJ0aGlua3BocCI%253D%26file%3D%26file%3D',
    'Hm_lpvt_9490413c5eebdadf757c2be2c816aedf': '16175430'
}

base_search = 'app="thinkphp" && country="CN"'

for page_num in range(1, 6):
    encoded_search = base64.b64encode(base_search.encode('utf-8')).decode('utf-8')
    url = f'https://fofa.so/result?page={page_num}&qbase64={encoded_search}'
    
    try:
        res = requests.get(url, headers={'Cookie': '; '.join([f'{k}={v}' for k, v in cookies.items()])}, timeout=5)
        if res.status_code != 200:
            continue
            
        doc = etree.HTML(res.content)
        links = doc.xpath('//div[@class="re-domain"]/a[@target="_blank"]/@href')
        
        if links:
            batch_output = '\n'.join(links)
            print(f'Page {page_num}: extracted {len(links)} URLs')
            
            with open('targets.txt', 'a', encoding='utf-8') as out:
                out.write(batch_output + '\n')
                
        time.sleep(1)  # Respect rate limits
        
    except (requests.RequestException, etree.ParserError):
        continue

Note: Cookie values must be updated manually before execution. FOFA’s free plan restricts pagination depth and may throttle repeated requests without delays.