feat: framework refactor + decouple from Hyperf#349
feat: framework refactor + decouple from Hyperf#349binaryfire wants to merge 3359 commits intohypervel:0.4from
Conversation
|
@albertcht To illustrate how much easier it will be to keep Hypervel in sync with Laravel after this refactor, I asked Claude how long it would take to merge laravel/framework#58461 (as an example) into this branch. This is what it said: So just 5-10 minutes of work with the help of AI tooling! Merging individual PRs is inefficient - merging releases would be better. I can set up a Discord channel where new releases are automatically posted via webhooks. Maybe someone in your team can be responsible for monitoring that channel's notifications and merging updates ever week or 2? I'll only be 1-2 hours of work once the codebases are 1:1. We should be diligent about staying on top of merging updates. Otherwise we'll end up in in the same as Hyperf - i.e. the codebase being completely out of date with the current Laravel API. |
|
Hi @binaryfire , Thank you for submitting this PR and for the detailed explanation of the refactor. After reading through it, I strongly agree that this is the best long-term direction for Hypervel. Refactoring Hypervel into a standalone framework and striving for 1:1 parity with Laravel will indeed solve the current issues regarding deep coupling with Hyperf, maintenance difficulties, outdated versions, and inefficient AI assistance. While this is a difficult step, it is absolutely necessary for the future of the project. Regarding this refactor and the planning for the v0.4 branch, I have a few thoughts to verify with you:
Thank you again for dedicating so much effort to driving this forward; this is a massive undertaking. Let's move forward gradually on this branch with ongoing Code Reviews. |
|
Hi @albertcht Thanks for the detailed response! I'm glad we're aligned on the direction. Let me address each point:
Let me know your thoughts! |
binaryfire
force-pushed
the
feature/hyperf-decouple
branch
from
8cec3bf to
bfffa6f
Compare
|
Hi @albertcht. The All the Laravel tests have been ported over and are passing (the unit tests, as well as the integration tests for MySQL, MariaDB, Postgres and SQLite). I've implemented Context-based coroutine safety, static caching for performance and modernised all the types. The code passes PHPStan level 5. Let me know if there's anything I've missed, if you have any ideas or you have any questions. The other packages aren't ready for review yet - many of them are mid-migration and contain temporary code. So please don't review the others yet :) I'll let you know when each one is ready. A few points:
|
binaryfire
force-pushed
the
feature/hyperf-decouple
branch
from
80f3ef2 to
94c115e
Compare
|
@albertcht The following packages are ready for review. I've modernised typing, optimised the code, added more tests (including integration tests) and fixed several bugs.
I've also ported https://github.com/friendsofhyperf/redis-subscriber into the Redis package. The subscription methods were all blocking - now they're coroutine friendly. With the previous implementation, if you wrapped
The approach follows the same pattern suggested in hyperf/hyperf#4775 (https://github.com/mix-php/redis-subscriber, which Deeka ported to https://github.com/friendsofhyperf/components). I.e. a dedicated raw socket connection with This is a good article re: this issue for reference: https://openswoole.com/article/redis-swoole-pubsub |
|
Hi @albertcht! The new This is Swoole-optimised version of Laravel's IoC Container, replacing Hyperf's container. The goal: give Hypervel the complete Laravel container API while maintaining performance parity with Hyperf's container and full coroutine safety for Swoole's long-running process model. Why replace Hyperf's container?Hyperf's container is minimal. It exposes
Also, the API is very different to Laravel's. This makes it difficult to port Laravel code or use Laravel's service provider patterns without shimming everything. The new container closes that gap completely and makes interacting with the container much more familiar to Laravel devs. It also means that our package and test code will be closer to 1:1 with Laravel now. APIThe new container implements the full Laravel container contract:
It also supports closure return-type bindings (register a binding by returning a typed value from a closure, including union types), Key API difference from HyperfLike Hyperf's Auto-singletoned instances are stored in a separate Attribute-based injection16 contextual attributes are included, providing declarative dependency injection:
Example: class OrderService
{
public function __construct(
#[Config('orders.tax_rate')] private float $taxRate,
#[Tag('payment-processors')] private array $processors,
#[Authenticated] private User $user,
) {}
}PerformanceBuild recipe cachingConstructor parameters are analyzed via reflection once per class and cached as Method parameter caching
Reflection caching
Hot-path optimizations
Performance vs HyperfThe singleton cache-hit path does marginally more work than Hyperf's single Coroutine safetyAll per-request state is stored in coroutine-local
Circular dependency detection uses two complementary mechanisms:
All transient Context state is cleaned up in Scoped instance cleanup is handled consistently across all invalidation paths. Tests~220 tests:
Everything passes at PHPStan level 5. Let me know what you think |
src/database/src/Listeners/RegisterConnectionResolverListener.php
Outdated
Show resolved
Hide resolved
… standardise conventions - Add native types to seed() in Contracts\TestCase and InteractsWithDatabase - Add native types to HandlesAssertions condition parameters - Fix Config::getPurgeAttributes() dead code after return statement - Fix Workbench::applicationUserModel() loose comparison (!= to !==) - Remove unsupported sqlsrv/MSSQL from SyncDatabaseEnvironmentVariables - Rename stubs/ directories to Fixtures/ for consistency across packages - Add missing method docblocks across 10 files
…methods to Gate Static caches (worker-lifetime): - Policy resolution cache: eliminates repeated reflection, class_exists probes, and subclass scans per model class - Guest-signature cache: eliminates repeated reflection for nullable parameter checks in canBeCalledWithUser() - Ability method name cache: eliminates repeated Str::camel() calls in formatAbilityToMethod() Code improvements: - check()/any(): plain foreach loops instead of Collection allocation - call_user_func: replaced with direct invocation in resolveUser() and guessPolicyName() - buildAbilityCallback(): string parsing moved to define() time, single func_get_args() call - resolvePolicyCallback(): formatAbilityToMethod() computed once, captured via use New feature — query-aware policy methods: - Gate::scope(): filters queries to authorized rows via optional policy *Scope methods - Gate::select(): returns SQL expressions for per-row authorization via optional *Select methods - Both support Gate before() callbacks and policy before() methods - Guest gating via canBeCalledWithUser() prevents TypeErrors for unauthenticated users - Contract interface updated with scope() and select() method signatures
getName() and getRecallerName() computed sha1(static::class) on every call. Both are called multiple times per auth flow. Since the guard name and class are immutable for the guard's lifetime, precompute once in the constructor.
When both user() and getPayload() are called in the same request, the JWT was decoded twice. JWT decoding involves base64 parsing, JSON deserialization, and signature verification. Cache the decoded payload in coroutine Context via Context::getOrSet() so it's computed once per token per request.
Each accessor (id(), token(), hash(), valid()) called explode() independently. During a typical remember-me flow, 4 separate explode operations ran on the same immutable string. Parse once in the constructor, reuse the cached array.
Previously, passing ['alias' => $expression] to select() or addSelect() silently dropped the alias key because only subqueries (via isQueryable()) were handled. Expressions were appended to $this->columns without aliasing. Now both methods check for ExpressionContract before isQueryable() and route aliased expressions through selectExpression(). In addSelect(), the existing guard for preserving default table.* columns is applied to expressions the same way it is for subqueries.
Provides assertScopeMatchesPolicy() and assertSelectMatchesPolicy() assertions that verify a policy's query-aware methods (*Scope, *Select) produce the same results as the per-instance PHP method. Catches drift between the dual definitions early in the test suite.
- Widen AbstractPaginator::appends() $value from ?string to array|string|null to match Hyperf/Laravel which accept array values for nested query parameters - Widen AbstractCursorPaginator::getCursorForItem() and getParametersForItem() $item from object to array|object — the method bodies already handle arrays via is_array() branches, and Laravel tests pass array items to cursor paginators - Add default value to AbstractCursorPaginator::$parameters (= []) to prevent "must not be accessed before initialization" when no parameters option is passed to the constructor
Port all 11 Laravel test files for the pagination package plus 2 fixture models. Includes tests for Cursor, CursorPaginator, LengthAwarePaginator, Paginator, UrlWindow, load morph chaining on both abstract paginators, and resource collection transformation. Merge 4 unique tests from Hyperf's LengthAwarePaginatorTest that cover nextPageUrl on last page, firstItem/lastItem offsets, appends with arrays, and toJson output verification. Adapt resource tests for Hypervel's TransformsToResourceCollection trait which uses guessResourceName() from the TransformsToResource trait on models.
The concrete Store already implements reflash(), but the Session contract was missing it. This caused phpstan errors when calling $request->session()->reflash() through the contract type.
RouteBinding::forModel() and related methods were typed with the concrete Container class instead of the contract. Only make() is called on the container, which is defined on the contract. Using the concrete type unnecessarily prevented passing mock containers in tests.
Marker interface matching Laravel's ShouldRescue. When a broadcast event implements this interface, dispatch and queue push operations are wrapped in rescue() to suppress exceptions.
- Add explicit route binding support via BindingRegistrar. The resolveBinding() method now tries explicit bindings first before falling back to implicit model binding. - Fix channelNameMatchesPattern() to escape literal dots in patterns before building the regex. Without this, dots matched any character. - Switch retrieveUser() to use $request->user() instead of resolving through the container auth manager directly. Both paths are coroutine-safe; aligning with Laravel simplifies upstream diffing. - Add binder() method using $this->container (DI pattern) instead of Laravel's Container::getInstance() static calls. - Update tests: port explicit binding and model route binding tests from Laravel, switch retrieveUser tests to $request->user() mocks, remove redundant flushChannels() from tearDown (handled by AfterEachTestSubscriber).
- Remove Pusher 6.x backwards-compat code: delete method_exists guard and manual HMAC fallback in resolveAuthenticatedUser(). Hypervel requires pusher/pusher-php-server ^7.2 which always has authenticateUser(). Keep json_decode since Pusher 7.x returns JSON strings. - Add JSONP callback support to decodePusherResponse(). When the request includes a callback parameter, return a JSONP response via response()->json()->withCallback(). - Switch auth() and validAuthenticationResponse() to pass $request to retrieveUser() matching the base class signature change. - Call authorizeChannel() and authorizePresenceChannel() directly without method_exists fallbacks to deprecated socket_auth() and presence_auth(). - Update tests to use $request->user() mocks, add JSONP test.
Switch auth() and validAuthenticationResponse() to pass $request to retrieveUser() matching the base class signature change. Update tests to use $request->user() mocks instead of container auth mocks.
- Switch auth() and validAuthenticationResponse() to pass $request to retrieveUser() matching the base class signature change. - Add cluster-safe broadcast path: use per-channel publish() when the connection is a Redis Cluster (via isCluster()), fall back to eval() with Lua script for non-cluster. Laravel uses class-based checks for PhpRedisClusterConnection/PredisClusterConnection which don't exist in Hypervel's single-connection Redis architecture. - Fix upstream socket duplication bug: Arr::pull() was called inside the array literal after $payload was already captured by 'data', causing socket to appear in both data.socket and top-level socket. Pull socket before building the array, matching the Pusher broadcaster's correct pattern. - Update tests to use $request->user() mocks, add cluster and non-cluster broadcast tests, add socket payload test.
BroadcastEvent: - Add ReadsQueueAttributes trait. Constructor now reads tries, timeout, backoff, maxExceptions, and deleteWhenMissingModels from PHP attributes via getAttributeValue(). - Add $deleteWhenMissingModels property (defaults to true). Pass the property default as fallback to getAttributeValue() to avoid TypeError when neither attribute nor property exists on the event. - Add getConnectionChannels() and getConnectionPayload() for per-connection channel and payload filtering in multi-connection broadcasts. - Add middleware() and failed() methods that proxy to the underlying event when those methods exist. - Reorder handle() to compute $name before $channels matching Laravel's ordering. UniqueBroadcastEvent: - Remove get_class($event) prefix from $uniqueId. UniqueLock:: getKey() already includes the class name via displayName(), so the prefix caused double-counting. - Initialize $uniqueId = '' and $uniqueFor = 0 to avoid uninitialized typed property issues. Port tests from Laravel: per-connection channels, middleware proxy, failed handler proxy, deleteWhenMissingModels default.
…Laravel AnonymousEvent: - Remove BroadcastManager constructor parameter. Use the broadcast() helper in send() instead, matching Laravel's pattern. - Update broadcastOn() return type to Channel|array. PendingBroadcast: - Normalize enums with enum_value() in via() before passing to broadcastVia(), matching Laravel's behavior. BroadcastController: - Add session reflash to both authenticate() and authenticateUser(). Without this, flash data is lost during AJAX broadcast auth. - Extend Hypervel\Routing\Controller for consistency with Laravel.
BroadcastManager: - Add ReadsQueueAttributes and ResolvesQueueRoutes traits for queue attribute and queue route resolution in queue(). - Add CachesRoutes check to routes() and userRoutes() to skip route registration when routes are cached. - Rewrite queue() with ShouldRescue support, queue attribute resolution, and queue route resolution for connection/queue. - Update mustBeUniqueAndCannotAcquireLock() to fall back to Cache repository when uniqueVia() doesn't exist. - Bind closure to $this in extend() so custom creators can access manager methods. - Add exception wrapping in doResolve() with connection name context. Wrapping is in doResolve() (not resolve()) because pooled drivers are created lazily via ObjectPool — a try/catch in resolve() would miss those failures. - Pass connection $name to doResolve() for error message context. - Add rescue() helper method for ShouldRescue support. - Update event() to use 'events' string key instead of the contract class, matching Laravel's canonical alias. - Update on() to not pass $this to AnonymousEvent constructor. BroadcastServiceProvider: - Add explicit BroadcastManager singleton binding matching Laravel. Update tests: lock key for UniqueBroadcastEvent, add route caching test, extend binding test, driver creation failure wrapping test.
Add testChannelNamePatternMatching test and the testChannelNameMatchesPattern helper method on the fake broadcaster class, matching Laravel's test coverage.
Move BroadcastManagerTest and SendingBroadcastsViaAnonymousEventTest to tests/Integration/Broadcasting/ matching Laravel's test directory structure. Both extend Testbench TestCase (integration tests). Port missing tests from Laravel: - testEventsCanBeBroadcastUsingQueueRoutes - testEventsCanBeRescued / testNowEventsCanBeRescued (ShouldRescue) - testUniqueEventsCanBeBroadcastWithUniqueIdFromProperty - testUniqueEventsCanBeBroadcastWithUniqueIdFromMethod Rename tests to match Laravel: - testExtendBindsCallbackToManager → testCustomDriverClosureBoundObjectIsBroadcastManager - testDriverCreationFailureWrapsExceptionWithConnectionName → testThrowExceptionWhenDriverCreationFails
2 participants
Hi @albertcht. This isn't ready yet but I'm opening it as a draft so we can begin discussions and code reviews. The goal of this PR is to refactor Hypervel to be a fully standalone framework that is as close to 1:1 parity with Laravel as possible.
Why one large PR
Sorry about the size of this PR. I tried spreading things across multiple branches but it made my work a lot more difficult. This is effectively a framework refactor - the database package is tightly coupled to many other packages (collections, pagination, pool) as well as several support classes, so all these things need to be updated together. Splitting it across branches would mean each branch needs multiple temporary workarounds + would have failing tests until merged together, making review and CI impractical.
A single large, reviewable PR is less risky than a stack of dependent branches that can't pass CI independently.
Reasons for the refactor
1. Outdated Hyperf packages
It's been difficult to migrate existing Laravel projects to Hypervel because Hyperf's database packages are quite outdated. There are almost 100 missing methods, missing traits, it doesn't support nested transactions, there are old Laravel bugs which haven't been fixed (eg. JSON indices aren't handled correctly), coroutine safety issues (eg. model
unguard(),withoutTouching()). Other packages like pagination, collections and support are outdated too.Stringablewas missing a bunch of methods and traits, for example. There are just too many to PR to Hyperf at this point.2. Faster framework development
We need to be able to move quickly and waiting for Hyperf maintainers to merge things adds a lot of friction to framework development. Decoupling means we don't need to work around things like PHP 8.4 compatibility while waiting for it to be added upstream. Hyperf's testing package uses PHPUnit 10 so we can't update to PHPUnit 13 (and Pest 4 in the skeleton) when it releases in a couple of weeks. v13 has the fix that allows
RunTestsInCoroutineto work with newer PHPUnit versions. There are lots of examples like this.3. Parity with Laravel
We need to avoid the same drift from Laravel that's happened with Hyperf since 2019. If we're not proactive with regularly merging Laravel updates every week we'll end up in the same situation. Having a 1:1 directory and code structure to Laravel whenever possible will make this much easier. Especially when using AI tools.
Most importantly, we need to make it easier for Laravel developers to use and contribute to the framework. That means following the same APIs and directory structures and only modifying code when there's a good reason to (coroutine safety, performance, type modernisation etc).
Right now the Hypervel codebase is confusing for both Laravel developers and AI tools:
hypervel/contractspackage, the Hyperf database code is split across 3 packages, the Hyperf pagination package ishyperf/paginatorand nothyperf/pagination)static::registerCallback('creating')vsstatic::creating())ConfigProviderand LaravelServiceProviderpatterns across different packages is confusing for anyone who doesn't know HyperfThis makes it difficult for Laravel developers to port over apps and to contribute to the framework.
4. AI
The above issues mean that AI needs a lot of guidance to understand the Hypervel codebase and generate Hypervel boilerplate. A few examples:
hypervel/contractsfor contracts) and then have to spend a lot of time grepping for things to find them.And so on... This greatly limits the effectiveness of building Hypervel apps with AI. Unfortunately MCP docs servers and CLAUDE.md rules don't solve all these problems - LLMs aren't great at following instructions well and the sheer volume of Laravel data they've trained on means they always default to Laravel-style code. The only solution is 1:1 parity. Small improvements such as adding native type hints are fine - models can solve that kind of thing quickly from exception messages.
What changed so far
New packages
illuminate/databaseportilluminate/collectionsportilluminate/paginationportilluminate/contracts)hyperf/pool)Macroableto a separate package for Laravel parityRemoved Hyperf dependencies so far
Database package
The big task was porting the database package, making it coroutine safe, implementing performance improvements like static caching and modernising the types.
whereLike,whereNot,groupLimit,rawValue,soleValue, JSON operations, etc.Collections package
Contracts package
Support package
hyperf/tappable,hyperf/stringable,hyperf/macroable,hyperf/codecdependenciesStr,Envand helper classes from LaravelHypervel\Contextwrappers (will be portinghyperf/contextsoon)Number::useCurrency()wasn't actually setting the currency)Coroutine safety
withoutEvents(),withoutBroadcasting(),withoutTouching()now use Context instead of static propertiesUnsetContextInTaskWorkerListenerto clear database context in task workersConnection::resetForPool()to prevent state leaks between coroutinesDatabaseTransactionsManagercoroutine-safeBenefits
Testing status so far
What's left (WIP)
The refactor process
Hyperf's Swoole packages like
pool,coroutine,contextandhttp-serverhaven't changed in many years so porting these is straightforward. A lot of the code can be simplified since we don't need SWOW support. And we can still support the ecosystem by contributing any improvements we make back to Hyperf in separate PRs.Eventually I'll refactor the bigger pieces like the container (contextual binding would be nice!) and the config system (completely drop
ConfigProviderand move entirely to service providers). But those will be future PRs. For now the main refactors are the database layer, collections and support classes + the simple Hyperf packages. I'll just port the container and config packages as-is for now.Let me know if you have any feedback, questions or suggestions. I'm happy to make any changes you want. I suggest we just work through this gradually, as an ongoing task over the next month or so. I'll continue working in this branch and ping you each time I add something new.
EDIT: New comments are getting lost in the commit history so linking them here:
Updated
hypervel/contextpackage ready for reviewSee: #349 (comment)
New
hypervel/http-server,hypervel/http&hypervel/routingpackages ready for reviewSee: #349 (comment)
New
hypervel/containerpackage ready for reviewSee: #349 (comment)
New
hypervel/context,hypervel/coordinator,hypervel/coroutine,hypervel/engine,hypervel/pool&hypervel/redispackages ready for reviewSee: #349 (comment)
New
hypervel/databasepackage ready for reviewSee: #349 (comment)