-
Notifications
You must be signed in to change notification settings - Fork 136
[test] e2e API test experiment #309
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
🧪 E2E Test Results❌ Some tests failed Summary
❌ Failed Testsmongodb (🌍 Community Worlds): e2e webhookWorkflowFile: redis (🌍 Community Worlds): e2e webhookWorkflowFile: starter (🌍 Community Worlds): e2e addTenWorkflowFile: starter (🌍 Community Worlds): e2e addTenWorkflowFile: starter (🌍 Community Worlds): e2e retryAttemptCounterWorkflowFile: starter (🌍 Community Worlds): e2e crossFileErrorWorkflow - stack traces work across imported modulesFile: starter (🌍 Community Worlds): e2e hookCleanupTestWorkflow - hook token reuse after workflow completionFile: starter (🌍 Community Worlds): e2e stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)File: starter (🌍 Community Worlds): e2e stepFunctionWithClosureWorkflow - step function with closure variables passed as argumentFile: starter (🌍 Community Worlds): e2e spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a stepFile: turso (🌍 Community Worlds): e2e addTenWorkflowFile: turso (🌍 Community Worlds): e2e addTenWorkflowFile: turso (🌍 Community Worlds): e2e should work with react rendering in stepFile: turso (🌍 Community Worlds): e2e promiseAllWorkflowFile: turso (🌍 Community Worlds): e2e promiseRaceWorkflowFile: turso (🌍 Community Worlds): e2e promiseAnyWorkflowFile: turso (🌍 Community Worlds): e2e readableStreamWorkflowFile: turso (🌍 Community Worlds): e2e hookWorkflowFile: turso (🌍 Community Worlds): e2e webhookWorkflowFile: turso (🌍 Community Worlds): e2e sleepingWorkflowFile: turso (🌍 Community Worlds): e2e nullByteWorkflowFile: turso (🌍 Community Worlds): e2e workflowAndStepMetadataWorkflowFile: turso (🌍 Community Worlds): e2e outputStreamWorkflowFile: turso (🌍 Community Worlds): e2e outputStreamInsideStepWorkflow - getWritable() called inside step functionsFile: turso (🌍 Community Worlds): e2e fetchWorkflowFile: turso (🌍 Community Worlds): e2e promiseRaceStressTestWorkflowFile: turso (🌍 Community Worlds): e2e retryAttemptCounterWorkflowFile: turso (🌍 Community Worlds): e2e retryableAndFatalErrorWorkflowFile: turso (🌍 Community Worlds): e2e crossFileErrorWorkflow - stack traces work across imported modulesFile: turso (🌍 Community Worlds): e2e hookCleanupTestWorkflow - hook token reuse after workflow completionFile: turso (🌍 Community Worlds): e2e stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)File: turso (🌍 Community Worlds): e2e stepFunctionWithClosureWorkflow - step function with closure variables passed as argumentFile: turso (🌍 Community Worlds): e2e closureVariableWorkflow - nested step functions with closure variablesFile: turso (🌍 Community Worlds): e2e spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a stepFile: astro (▲ Vercel Production): e2e addTenWorkflowFile: astro (▲ Vercel Production): e2e addTenWorkflowFile: astro (▲ Vercel Production): e2e retryAttemptCounterWorkflowFile: astro (▲ Vercel Production): e2e crossFileErrorWorkflow - stack traces work across imported modulesFile: astro (▲ Vercel Production): e2e hookCleanupTestWorkflow - hook token reuse after workflow completionFile: astro (▲ Vercel Production): e2e stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)File: astro (▲ Vercel Production): e2e stepFunctionWithClosureWorkflow - step function with closure variables passed as argumentFile: astro (▲ Vercel Production): e2e spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a stepFile: example (▲ Vercel Production): e2e addTenWorkflowFile: example (▲ Vercel Production): e2e addTenWorkflowFile: example (▲ Vercel Production): e2e retryAttemptCounterWorkflowFile: example (▲ Vercel Production): e2e crossFileErrorWorkflow - stack traces work across imported modulesFile: example (▲ Vercel Production): e2e hookCleanupTestWorkflow - hook token reuse after workflow completionFile: example (▲ Vercel Production): e2e stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)File: example (▲ Vercel Production): e2e stepFunctionWithClosureWorkflow - step function with closure variables passed as argumentFile: example (▲ Vercel Production): e2e spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a stepFile: express (▲ Vercel Production): e2e addTenWorkflowFile: express (▲ Vercel Production): e2e addTenWorkflowFile: express (▲ Vercel Production): e2e retryAttemptCounterWorkflowFile: express (▲ Vercel Production): e2e crossFileErrorWorkflow - stack traces work across imported modulesFile: express (▲ Vercel Production): e2e hookCleanupTestWorkflow - hook token reuse after workflow completionFile: express (▲ Vercel Production): e2e stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)File: express (▲ Vercel Production): e2e stepFunctionWithClosureWorkflow - step function with closure variables passed as argumentFile: express (▲ Vercel Production): e2e spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a stepFile: e2e-vercel-prod-fastify (📋 Other): e2e addTenWorkflowFile: e2e-vercel-prod-fastify (📋 Other): e2e addTenWorkflowFile: e2e-vercel-prod-fastify (📋 Other): e2e retryAttemptCounterWorkflowFile: e2e-vercel-prod-fastify (📋 Other): e2e crossFileErrorWorkflow - stack traces work across imported modulesFile: e2e-vercel-prod-fastify (📋 Other): e2e hookCleanupTestWorkflow - hook token reuse after workflow completionFile: e2e-vercel-prod-fastify (📋 Other): e2e stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)File: e2e-vercel-prod-fastify (📋 Other): e2e stepFunctionWithClosureWorkflow - step function with closure variables passed as argumentFile: e2e-vercel-prod-fastify (📋 Other): e2e spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a stepFile: hono (▲ Vercel Production): e2e addTenWorkflowFile: hono (▲ Vercel Production): e2e addTenWorkflowFile: hono (▲ Vercel Production): e2e retryAttemptCounterWorkflowFile: hono (▲ Vercel Production): e2e crossFileErrorWorkflow - stack traces work across imported modulesFile: hono (▲ Vercel Production): e2e hookCleanupTestWorkflow - hook token reuse after workflow completionFile: hono (▲ Vercel Production): e2e stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)File: hono (▲ Vercel Production): e2e stepFunctionWithClosureWorkflow - step function with closure variables passed as argumentFile: hono (▲ Vercel Production): e2e spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a stepFile: nextjs-turbopack (▲ Vercel Production): e2e addTenWorkflowFile: nextjs-turbopack (▲ Vercel Production): e2e addTenWorkflowFile: nextjs-turbopack (▲ Vercel Production): e2e retryAttemptCounterWorkflowFile: nextjs-turbopack (▲ Vercel Production): e2e crossFileErrorWorkflow - stack traces work across imported modulesFile: nextjs-turbopack (▲ Vercel Production): e2e hookCleanupTestWorkflow - hook token reuse after workflow completionFile: nextjs-turbopack (▲ Vercel Production): e2e stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)File: nextjs-turbopack (▲ Vercel Production): e2e stepFunctionWithClosureWorkflow - step function with closure variables passed as argumentFile: nextjs-turbopack (▲ Vercel Production): e2e spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a stepFile: nextjs-webpack (▲ Vercel Production): e2e addTenWorkflowFile: nextjs-webpack (▲ Vercel Production): e2e addTenWorkflowFile: nextjs-webpack (▲ Vercel Production): e2e retryAttemptCounterWorkflowFile: nextjs-webpack (▲ Vercel Production): e2e crossFileErrorWorkflow - stack traces work across imported modulesFile: nextjs-webpack (▲ Vercel Production): e2e hookCleanupTestWorkflow - hook token reuse after workflow completionFile: nextjs-webpack (▲ Vercel Production): e2e stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)File: nextjs-webpack (▲ Vercel Production): e2e stepFunctionWithClosureWorkflow - step function with closure variables passed as argumentFile: nextjs-webpack (▲ Vercel Production): e2e spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a stepFile: nitro (▲ Vercel Production): e2e addTenWorkflowFile: nitro (▲ Vercel Production): e2e addTenWorkflowFile: nitro (▲ Vercel Production): e2e retryAttemptCounterWorkflowFile: nitro (▲ Vercel Production): e2e crossFileErrorWorkflow - stack traces work across imported modulesFile: nitro (▲ Vercel Production): e2e hookCleanupTestWorkflow - hook token reuse after workflow completionFile: nitro (▲ Vercel Production): e2e stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)File: nitro (▲ Vercel Production): e2e stepFunctionWithClosureWorkflow - step function with closure variables passed as argumentFile: nitro (▲ Vercel Production): e2e spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a stepFile: nuxt (▲ Vercel Production): e2e addTenWorkflowFile: nuxt (▲ Vercel Production): e2e addTenWorkflowFile: nuxt (▲ Vercel Production): e2e retryAttemptCounterWorkflowFile: nuxt (▲ Vercel Production): e2e crossFileErrorWorkflow - stack traces work across imported modulesFile: nuxt (▲ Vercel Production): e2e hookCleanupTestWorkflow - hook token reuse after workflow completionFile: nuxt (▲ Vercel Production): e2e stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)File: nuxt (▲ Vercel Production): e2e stepFunctionWithClosureWorkflow - step function with closure variables passed as argumentFile: nuxt (▲ Vercel Production): e2e spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a stepFile: sveltekit (▲ Vercel Production): e2e addTenWorkflowFile: sveltekit (▲ Vercel Production): e2e addTenWorkflowFile: sveltekit (▲ Vercel Production): e2e retryAttemptCounterWorkflowFile: sveltekit (▲ Vercel Production): e2e crossFileErrorWorkflow - stack traces work across imported modulesFile: sveltekit (▲ Vercel Production): e2e hookCleanupTestWorkflow - hook token reuse after workflow completionFile: sveltekit (▲ Vercel Production): e2e stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)File: sveltekit (▲ Vercel Production): e2e stepFunctionWithClosureWorkflow - step function with closure variables passed as argumentFile: sveltekit (▲ Vercel Production): e2e spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a stepFile: vite (▲ Vercel Production): e2e addTenWorkflowFile: vite (▲ Vercel Production): e2e addTenWorkflowFile: vite (▲ Vercel Production): e2e retryAttemptCounterWorkflowFile: vite (▲ Vercel Production): e2e crossFileErrorWorkflow - stack traces work across imported modulesFile: vite (▲ Vercel Production): e2e hookCleanupTestWorkflow - hook token reuse after workflow completionFile: vite (▲ Vercel Production): e2e stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)File: vite (▲ Vercel Production): e2e stepFunctionWithClosureWorkflow - step function with closure variables passed as argumentFile: vite (▲ Vercel Production): e2e spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a stepFile: Details by Category❌ ▲ Vercel Production
✅ 🪟 Windows
❌ 🌍 Community Worlds
❌ 📋 Other
Check the workflow run for details. |
| const defaultUrl = | ||
| 'https://workflow-server-git-peter-fix-hono-metrics.vercel.sh/api'; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| const defaultUrl = | |
| 'https://workflow-server-git-peter-fix-hono-metrics.vercel.sh/api'; | |
| const defaultUrl = 'https://vercel-workflow.com/api'; |
The default API endpoint for Vercel workflow operations has been incorrectly set to a personal development preview deployment URL instead of the production endpoint, which will cause all workflow API operations to fail in production environments.
View Details
Analysis
Hardcoded Development Preview Deployment URL Breaks Production Vercel Workflows
What fails: getHttpUrl() in packages/world-vercel/src/utils.ts uses a personal Vercel preview deployment URL (https://workflow-server-git-peter-fix-hono-metrics.vercel.sh/api) instead of the production endpoint, causing all workflow API operations to fail in production environments.
How to trigger it: Call any workflow API function that uses the default configuration without providing a custom baseUrl or projectId/teamId (e.g., createWorkflowRun(), getWorkflowRun(), createHook(), deleteHook()). These will attempt to reach the personal preview deployment which will not be available in production, resulting in connection errors.
What happens vs expected:
- Current: All workflow API requests fail with connection errors to
https://workflow-server-git-peter-fix-hono-metrics.vercel.sh/api, a personal development deployment that will not be accessible in production - Expected: Requests should use the production endpoint
https://vercel-workflow.com/apias established in commit 2dca0d4
Verification: The correct production URL was confirmed in git history at commit 2dca0d4 ([world-vercel] Use custom user-agent for API requests), where the defaultUrl was properly set to https://vercel-workflow.com/api.
This is #232 but with e2e tests against a newer API endpoint