Skip to content

README.md

Latest commit

 

History

History
700 lines (513 loc) · 26.8 KB

README.md

File metadata and controls

700 lines (513 loc) · 26.8 KB
Raw

TwexAPI Python SDK: Twitter API for search, followers, DMs & X automation

Use the TwexAPI Python SDK to search tweets, scrape Twitter followers, and read X profiles, timelines, replies, and threads. Send DMs, search communities, fetch lists, articles, hashtags, cashtags, and global trending tweets with typed request objects. Like, retweet, follow, and post through documented REST routes. It is a Twitter API alternative for apps, scripts, and agents.

REST API | TypeScript SDK | Ruby SDK | Dashboard

Speakeasy generates this SDK.

Common Twitter & X tasks

Task REST Route Usage
Search tweets without the X API POST /twitter/advanced_search/page Use keyword queries and paginate with a cursor.
Search hashtags or cashtags POST /twitter/hashtags, POST /twitter/cashtags Filter by tag and sort order.
Read an X profile GET /twitter/{screen_name}/about Look up a user by screen name.
Read a profile timeline GET /twitter/{screen_name}/timeline/page Paginate bounded results.
Scrape Twitter followers POST /v3/twitter/users/followers Use the v3 follower list.
Scrape following accounts POST /v3/twitter/users/following Use the v3 following list.
Read tweet replies POST /twitter/tweets/{tweet_id}/replies/page Paginate replies by tweet id.
Read a tweet thread POST /twitter/tweets/thread_by_id Fetch the thread from a root tweet.
Send or read DMs /v3/twitter/send-dm, /v3/twitter/dm-history Use v3 XChat endpoints.
Search communities POST /twitter/community/search Find communities, then load tweets or members.
Get global trending tweets GET /twitter/global-trending/tweets Filter by country, topic, and content.
Post or reply POST /twitter/tweets/create Confirm the account cookie and payload.

Package & registry trust

Summary

X API Scraper: Speakeasy-ready OpenAPI document for the x-api-scraper TypeScript SDK.

The SDK wraps TwexAPI's X/Twitter API surface with bearer-token authentication. Cookie/proxy based endpoints are excluded except tweet actions, follow/unfollow, and v3 DM operations. Paid engagement services, profile mutation, legacy-only operations, and non-v3 follower/following endpoints remain excluded.

Not affiliated with X Corp.

Table of Contents

SDK Installation

Requires Python 3.10 or higher.

pip install x_api_scraper

Or install from GitHub until the package is published:

pip install git+https://github.com/twexapi-dev/x-api-scraper-python.git

uv and poetry also work:

uv add x_api_scraper
poetry add x_api_scraper

Shell and script usage with uv

You can use this SDK in a Python shell with uv and the uvx command that comes with it like so:

uvx --from x_api_scraper python

It's also possible to write a standalone Python script without needing to set up a whole project like so:

#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.10"
# dependencies = [
#     "x_api_scraper",
# ]
# ///

from x_api_scraper import XapiScraper

sdk = XapiScraper(
  # SDK arguments
)

# Rest of script here...

Once that is saved to a file, you can run it with uv run script.py where script.py can be replaced with the actual file name.

IDE Support

PyCharm

Generally, the SDK will work well with most IDEs out of the box. However, when using PyCharm, you can enjoy much better integration with Pydantic by installing an additional plugin.

SDK Example Usage

Example

import os
from x_api_scraper import XapiScraper

with XapiScraper(
    bearer_auth=os.getenv("X_API_SCRAPER_KEY"),
) as client:
    res = client.search.advanced(
        search_terms=["from:elonmusk"],
        sort_by="Latest",
        next_cursor="",
    )
    print(res)

    about = client.users.get_about(screen_name="elonmusk")
    followers = client.users.followers.list(username="elonmusk")

Get an API key from the TwexAPI dashboard. Pass it as bearer_auth, or set X_API_SCRAPER_KEY / X_API_SCRAPER_BEARER_AUTH.

Write actions (tweet, follow, like, DM send) also need a Twitter cookie or auth_token on the request. Pass them on the operation input.

Keep API keys out of source code, URLs, and logs.


The same SDK client can also be used to make asynchronous requests by importing asyncio.

# Asynchronous Example
import asyncio
import os
from x_api_scraper import XapiScraper

async def main():

    async with XapiScraper(
        bearer_auth=os.getenv("X_API_SCRAPER_BEARER_AUTH", ""),
    ) as xapi_scraper:

        res = await xapi_scraper.search.advanced_async(search_terms=[
            "<value 1>",
            "<value 2>",
            "<value 3>",
        ], sort_by="<value>", next_cursor="")

        # Handle response
        print(res)

asyncio.run(main())

Authentication

Per-Client Security Schemes

This SDK supports the following security scheme globally:

Name Type Scheme Environment Variable
bearer_auth http HTTP Bearer X_API_SCRAPER_BEARER_AUTH

To authenticate with the API the bearer_auth parameter must be set when initializing the SDK client instance. For example:

import os
from x_api_scraper import XapiScraper


with XapiScraper(
    bearer_auth=os.getenv("X_API_SCRAPER_BEARER_AUTH", ""),
) as xapi_scraper:

    res = xapi_scraper.search.advanced(search_terms=[
        "<value 1>",
        "<value 2>",
        "<value 3>",
    ], sort_by="<value>", next_cursor="")

    # Handle response
    print(res)

Available Resources and Operations

Available methods

Account

Analysis

Articles

  • fetch - Batch Fetch X Articles
  • markdown - Fetch Article as Markdown

Communities

Dm

Lists

Search

Timelines

Trending

Tweets

Tweets.Actions

Tweets.Engagement

Tweets.Replies

  • page - Get Replies by Page

Users

Users.Followers

  • list - Get Followers (v3)
  • verified - Get Verified Followers

Users.Following

  • list - Get Following (v3)

Retries

Some of the endpoints in this SDK support retries. If you use the SDK without any configuration, it will fall back to the default retry strategy provided by the API. However, the default retry strategy can be overridden on a per-operation basis, or across the entire SDK.

To change the default retry strategy for a single API call, simply provide a RetryConfig object to the call:

import os
from x_api_scraper import XapiScraper
from x_api_scraper.utils import BackoffStrategy, RetryConfig


with XapiScraper(
    bearer_auth=os.getenv("X_API_SCRAPER_BEARER_AUTH", ""),
) as xapi_scraper:

    res = xapi_scraper.search.advanced(search_terms=[
        "<value 1>",
        "<value 2>",
        "<value 3>",
    ], sort_by="<value>", next_cursor="",
        RetryConfig("backoff", BackoffStrategy(1, 50, 1.1, 100), False))

    # Handle response
    print(res)

If you'd like to override the default retry strategy for all operations that support retries, you can use the retry_config optional parameter when initializing the SDK:

import os
from x_api_scraper import XapiScraper
from x_api_scraper.utils import BackoffStrategy, RetryConfig


with XapiScraper(
    retry_config=RetryConfig("backoff", BackoffStrategy(1, 50, 1.1, 100), False),
    bearer_auth=os.getenv("X_API_SCRAPER_BEARER_AUTH", ""),
) as xapi_scraper:

    res = xapi_scraper.search.advanced(search_terms=[
        "<value 1>",
        "<value 2>",
        "<value 3>",
    ], sort_by="<value>", next_cursor="")

    # Handle response
    print(res)

Error Handling

XapiScraperError is the base class for all HTTP error responses. It has the following properties:

Property Type Description
err.message str Error message
err.status_code int HTTP response status code eg 404
err.headers httpx.Headers HTTP response headers
err.body str HTTP body. Can be empty string if no body is returned.
err.raw_response httpx.Response Raw HTTP response
err.data Optional. Some errors may contain structured data. See Error Classes.

Example

import os
from x_api_scraper import XapiScraper, errors


with XapiScraper(
    bearer_auth=os.getenv("X_API_SCRAPER_BEARER_AUTH", ""),
) as xapi_scraper:
    res = None
    try:

        res = xapi_scraper.search.advanced(search_terms=[
            "<value 1>",
            "<value 2>",
            "<value 3>",
        ], sort_by="<value>", next_cursor="")

        # Handle response
        print(res)


    except errors.XapiScraperError as e:
        # The base class for HTTP error responses
        print(e.message)
        print(e.status_code)
        print(e.body)
        print(e.headers)
        print(e.raw_response)

        # Depending on the method different errors may be thrown
        if isinstance(e, errors.HTTPValidationError):
            print(e.data.detail)  # Optional[List[models.ValidationError]]

Error Classes

Primary errors:

Less common errors (5)

Network errors:

Inherit from XapiScraperError:

  • ResponseValidationError: Type mismatch between the response data and the expected Pydantic model. Provides access to the Pydantic validation error via the cause attribute.

* Check the method documentation to see if the error is applicable.

Server Selection

Select Server by Name

You can override the default server globally by passing a server name to the server: str optional parameter when initializing the SDK client instance. The selected server will then be used as the default on the operations that use it. This table lists the names associated with the available servers:

Name Server Description
production https://api.twexapi.io TwexAPI production API

Example

import os
from x_api_scraper import XapiScraper


with XapiScraper(
    server="production",
    bearer_auth=os.getenv("X_API_SCRAPER_BEARER_AUTH", ""),
) as xapi_scraper:

    res = xapi_scraper.search.advanced(search_terms=[
        "<value 1>",
        "<value 2>",
        "<value 3>",
    ], sort_by="<value>", next_cursor="")

    # Handle response
    print(res)

Override Server URL Per-Client

The default server can also be overridden globally by passing a URL to the server_url: str optional parameter when initializing the SDK client instance. For example:

import os
from x_api_scraper import XapiScraper


with XapiScraper(
    server_url="https://api.twexapi.io",
    bearer_auth=os.getenv("X_API_SCRAPER_BEARER_AUTH", ""),
) as xapi_scraper:

    res = xapi_scraper.search.advanced(search_terms=[
        "<value 1>",
        "<value 2>",
        "<value 3>",
    ], sort_by="<value>", next_cursor="")

    # Handle response
    print(res)

Custom HTTP Client

The Python SDK makes API calls using the httpx HTTP library. In order to provide a convenient way to configure timeouts, cookies, proxies, custom headers, and other low-level configuration, you can initialize the SDK client with your own HTTP client instance. Depending on whether you are using the sync or async version of the SDK, you can pass an instance of HttpClient or AsyncHttpClient respectively, which are Protocol's ensuring that the client has the necessary methods to make API calls. This allows you to wrap the client with your own custom logic, such as adding custom headers, logging, or error handling, or you can just pass an instance of httpx.Client or httpx.AsyncClient directly.

For example, you could specify a header for every request that this sdk makes as follows:

from x_api_scraper import XapiScraper
import httpx

http_client = httpx.Client(headers={"x-custom-header": "someValue"})
s = XapiScraper(client=http_client)

or you could wrap the client with your own custom logic:

from x_api_scraper import XapiScraper
from x_api_scraper.httpclient import AsyncHttpClient
import httpx

class CustomClient(AsyncHttpClient):
    client: AsyncHttpClient

    def __init__(self, client: AsyncHttpClient):
        self.client = client

    async def send(
        self,
        request: httpx.Request,
        *,
        stream: bool = False,
        auth: Union[
            httpx._types.AuthTypes, httpx._client.UseClientDefault, None
        ] = httpx.USE_CLIENT_DEFAULT,
        follow_redirects: Union[
            bool, httpx._client.UseClientDefault
        ] = httpx.USE_CLIENT_DEFAULT,
    ) -> httpx.Response:
        request.headers["Client-Level-Header"] = "added by client"

        return await self.client.send(
            request, stream=stream, auth=auth, follow_redirects=follow_redirects
        )

    def build_request(
        self,
        method: str,
        url: httpx._types.URLTypes,
        *,
        content: Optional[httpx._types.RequestContent] = None,
        data: Optional[httpx._types.RequestData] = None,
        files: Optional[httpx._types.RequestFiles] = None,
        json: Optional[Any] = None,
        params: Optional[httpx._types.QueryParamTypes] = None,
        headers: Optional[httpx._types.HeaderTypes] = None,
        cookies: Optional[httpx._types.CookieTypes] = None,
        timeout: Union[
            httpx._types.TimeoutTypes, httpx._client.UseClientDefault
        ] = httpx.USE_CLIENT_DEFAULT,
        extensions: Optional[httpx._types.RequestExtensions] = None,
    ) -> httpx.Request:
        return self.client.build_request(
            method,
            url,
            content=content,
            data=data,
            files=files,
            json=json,
            params=params,
            headers=headers,
            cookies=cookies,
            timeout=timeout,
            extensions=extensions,
        )

s = XapiScraper(async_client=CustomClient(httpx.AsyncClient()))

httpx2 (Pydantic's httpx fork)

httpx2 is Pydantic's maintained fork of httpx. To run this SDK on httpx2, call alias_httpx() at your program's entry point, before importing the SDK, so every import httpx — including the ones inside the SDK — resolves to httpx2:

import httpx2

httpx2.alias_httpx()

from x_api_scraper import XapiScraper

s = XapiScraper()

An SDK can also be generated against httpx2 directly, so it depends on the fork instead of httpx, by setting python.httpClientLibrary: httpx2 in gen.yaml.

Resource Management

The XapiScraper class implements the context manager protocol and registers a finalizer function to close the underlying sync and async HTTPX clients it uses under the hood. This will close HTTP connections, release memory and free up other resources held by the SDK. In short-lived Python programs and notebooks that make a few SDK method calls, resource management may not be a concern. However, in longer-lived programs, it is beneficial to create a single SDK instance via a context manager and reuse it across the application.

import os
from x_api_scraper import XapiScraper
def main():

    with XapiScraper(
        bearer_auth=os.getenv("X_API_SCRAPER_BEARER_AUTH", ""),
    ) as xapi_scraper:
        # Rest of application here...


# Or when using async:
async def amain():

    async with XapiScraper(
        bearer_auth=os.getenv("X_API_SCRAPER_BEARER_AUTH", ""),
    ) as xapi_scraper:
        # Rest of application here...

Debugging

You can setup your SDK to emit debug logs for SDK requests and responses.

You can pass your own logger class directly into your SDK.

from x_api_scraper import XapiScraper
import logging

logging.basicConfig(level=logging.DEBUG)
s = XapiScraper(debug_logger=logging.getLogger("x_api_scraper"))

You can also enable a default debug logger by setting an environment variable X_API_SCRAPER_DEBUG to true.

Development

Maturity

This SDK is in beta, and there may be breaking changes between versions without a major version update. Therefore, we recommend pinning usage to a specific package version. This way, you can install the same version each time without breaking changes unless you are intentionally looking for the latest version.

Contributions

While we value open-source contributions to this SDK, this library is generated programmatically. Any manual changes added to internal files will be overwritten on the next generation. We look forward to hearing your feedback. Feel free to open a PR or an issue with a proof of concept and we'll do our best to include it in a future release.

To regenerate from the OpenAPI document:

speakeasy run

The source spec lives in openapi.sdk.json. Keep it in sync with x-api-scraper-typescript.

GitHub Actions can regenerate the SDK and publish to PyPI. Add these repository secrets first:

  • SPEAKEASY_API_KEY
  • PYPI_TOKEN

TwexAPI is an independent third-party service. Not affiliated with X Corp. "Twitter" and "X" are trademarks of X Corp.

SDK Created by Speakeasy