|
| 1 | +# Apache Hadoop YARN Capacity Scheduler UI |
| 2 | + |
| 3 | +A modern React-based web interface for managing and configuring the YARN Capacity Scheduler. |
| 4 | + |
| 5 | +## Overview |
| 6 | + |
| 7 | +The YARN Capacity Scheduler UI provides an intuitive graphical interface for: |
| 8 | +- **Queue Management**: Visual queue hierarchy with drag-and-drop organization |
| 9 | +- **Capacity Configuration**: Interactive capacity allocation and resource management |
| 10 | +- **Node Labels**: Partition management and node-to-label assignments |
| 11 | +- **Placement Rules**: Configure application placement policies |
| 12 | +- **Global Settings**: Manage scheduler-wide configuration parameters |
| 13 | +- **Configuration Validation**: Real-time validation with helpful error messages |
| 14 | +- **Change Staging**: Review and apply configuration changes in batches |
| 15 | + |
| 16 | +## Technology Stack |
| 17 | + |
| 18 | +- **Framework**: React 19 with TypeScript (strict mode) |
| 19 | +- **Build Tool**: Vite 6.4 with React Router v7 (SPA mode) |
| 20 | +- **UI Components**: shadcn/ui, Radix UI, Tailwind CSS |
| 21 | +- **State Management**: Zustand with Immer |
| 22 | +- **Visualization**: XYFlow (React Flow) for queue hierarchy |
| 23 | +- **Testing**: Vitest + React Testing Library |
| 24 | + |
| 25 | +## Building |
| 26 | + |
| 27 | +### Prerequisites |
| 28 | + |
| 29 | +- **Maven**: 3.6.0 or later |
| 30 | +- **Node.js**: 22.16.0+ (automatically installed by Maven during build) |
| 31 | +- **Java**: JDK 8 or later |
| 32 | + |
| 33 | +### Maven Build |
| 34 | + |
| 35 | +Build the UI as part of the YARN build using the `yarn-ui` profile: |
| 36 | + |
| 37 | +```bash |
| 38 | +# From hadoop-yarn directory |
| 39 | +cd hadoop-yarn-project/hadoop-yarn |
| 40 | +mvn clean package -Pyarn-ui |
| 41 | + |
| 42 | +# From hadoop-yarn-capacity-scheduler-ui directory |
| 43 | +cd hadoop-yarn-project/hadoop-yarn/hadoop-yarn-capacity-scheduler-ui |
| 44 | +mvn clean package -Pyarn-ui |
| 45 | +``` |
| 46 | + |
| 47 | +This will: |
| 48 | +1. Install Node.js 22.16.0 and npm locally in `target/webapp/node/` |
| 49 | +2. Install npm dependencies |
| 50 | +3. Build the React application |
| 51 | +4. Package everything into a WAR file at `target/hadoop-yarn-capacity-scheduler-ui-*.war` |
| 52 | + |
| 53 | +### Build Output |
| 54 | + |
| 55 | +The build creates: |
| 56 | +- `target/webapp/build/client/` - Built React application (static files) |
| 57 | +- `target/hadoop-yarn-capacity-scheduler-ui-*.war` - Deployable WAR file |
| 58 | + |
| 59 | +## Development |
| 60 | + |
| 61 | +### Local Development Setup |
| 62 | + |
| 63 | +```bash |
| 64 | +cd src/main/webapp |
| 65 | + |
| 66 | +# Install dependencies |
| 67 | +npm install |
| 68 | + |
| 69 | +# Start development server |
| 70 | +npm run dev |
| 71 | +``` |
| 72 | + |
| 73 | +The development server runs at `http://localhost:5173` with hot module replacement. |
| 74 | + |
| 75 | +### Environment Variables |
| 76 | + |
| 77 | +Create a `.env` file in `src/main/webapp/` based on `.env.example`: |
| 78 | + |
| 79 | +```bash |
| 80 | +# Mock mode: "static" (use mock data), "cluster" (proxy to real cluster), "off" (no mocking) |
| 81 | +VITE_API_MOCK_MODE=static |
| 82 | + |
| 83 | +# YARN ResourceManager URL (required when VITE_API_MOCK_MODE=cluster) |
| 84 | +VITE_CLUSTER_PROXY_TARGET=http://localhost:8088 |
| 85 | + |
| 86 | +# Development flags |
| 87 | +VITE_READONLY_MODE=false |
| 88 | +VITE_YARN_USER_NAME=admin |
| 89 | +``` |
| 90 | + |
| 91 | +### Available Scripts |
| 92 | + |
| 93 | +```bash |
| 94 | +npm run dev # Start development server |
| 95 | +npm run build # Production build |
| 96 | +npm run start # Preview production build |
| 97 | +npm run test # Run tests |
| 98 | +npm run test:ui # Run tests with UI |
| 99 | +npm run test:coverage # Generate test coverage report |
| 100 | +npm run lint # Lint code |
| 101 | +npm run lint:fix # Fix linting issues |
| 102 | +npm run format # Format code with Prettier |
| 103 | +``` |
| 104 | + |
| 105 | +### Running Tests |
| 106 | + |
| 107 | +```bash |
| 108 | +# Run all tests |
| 109 | +npm test |
| 110 | + |
| 111 | +# Run tests in watch mode |
| 112 | +npm run test |
| 113 | + |
| 114 | +# Run tests with coverage |
| 115 | +npm run test:coverage |
| 116 | + |
| 117 | +# Run tests with UI |
| 118 | +npm run test:ui |
| 119 | +``` |
| 120 | + |
| 121 | +## Deployment |
| 122 | + |
| 123 | +### WAR Deployment |
| 124 | + |
| 125 | +The built WAR file can be deployed to a servlet container (Tomcat, Jetty, etc.) or integrated into the YARN ResourceManager web application. |
| 126 | + |
| 127 | +### API Endpoints |
| 128 | + |
| 129 | +The UI requires access to the following YARN ResourceManager REST API endpoints: |
| 130 | + |
| 131 | +- `GET /ws/v1/cluster/scheduler` - Get scheduler configuration |
| 132 | +- `PUT /ws/v1/cluster/scheduler-conf` - Update scheduler configuration |
| 133 | +- `GET /ws/v1/cluster/scheduler-conf` - Get mutable configuration |
| 134 | +- `GET /ws/v1/cluster/nodes` - Get cluster nodes |
| 135 | +- `GET /conf` - Get Hadoop configuration |
| 136 | + |
| 137 | +### Integration with ResourceManager |
| 138 | + |
| 139 | +When deployed alongside the ResourceManager, the UI can access these endpoints directly. The `web.xml` includes SPA routing configuration to handle client-side routing. |
| 140 | + |
| 141 | +## Architecture |
| 142 | + |
| 143 | +### Directory Structure |
| 144 | + |
| 145 | +``` |
| 146 | +src/main/webapp/ |
| 147 | +├── src/ |
| 148 | +│ ├── app/ # React Router application setup |
| 149 | +│ ├── components/ # Reusable UI components |
| 150 | +│ ├── config/ # Scheduler configuration schemas |
| 151 | +│ ├── contexts/ # React contexts |
| 152 | +│ ├── features/ # Feature modules |
| 153 | +│ │ ├── global-settings/ |
| 154 | +│ │ ├── node-labels/ |
| 155 | +│ │ ├── placement-rules/ |
| 156 | +│ │ ├── property-editor/ |
| 157 | +│ │ ├── queue-management/ |
| 158 | +│ │ ├── staged-changes/ |
| 159 | +│ │ ├── template-config/ |
| 160 | +│ │ └── validation/ |
| 161 | +│ ├── hooks/ # Custom React hooks |
| 162 | +│ ├── lib/ # API client, utilities |
| 163 | +│ ├── stores/ # Zustand stores |
| 164 | +│ ├── types/ # TypeScript type definitions |
| 165 | +│ └── utils/ # Helper functions |
| 166 | +├── public/ # Static assets |
| 167 | +└── WEB-INF/ # Web application descriptor |
| 168 | +``` |
| 169 | + |
| 170 | +### State Management |
| 171 | + |
| 172 | +The application uses Zustand for global state management with the following stores: |
| 173 | + |
| 174 | +- **SchedulerStore**: Manages scheduler configuration and queue hierarchy |
| 175 | +- **NodeLabelStore**: Handles node labels and partitions |
| 176 | +- **StagedChangesStore**: Tracks pending configuration changes |
| 177 | +- **ValidationStore**: Manages validation state and error messages |
| 178 | + |
| 179 | +## Configuration |
| 180 | + |
| 181 | +### Build Profiles |
| 182 | + |
| 183 | +- **yarn-ui**: Production build (default profile in this module) |
| 184 | +- **yarn-ui-dev**: Development build (includes mock data in WAR) |
| 185 | + |
| 186 | +### Maven Properties |
| 187 | + |
| 188 | +- `packagingType`: WAR when profile is active, POM otherwise |
| 189 | +- `webappDir`: Build directory (`${basedir}/target/webapp`) |
| 190 | +- `nodeExecutable`: Path to locally installed Node.js |
| 191 | +- `keepUIBuildCache`: Set to `true` to preserve node_modules between builds |
| 192 | + |
| 193 | +## Browser Support |
| 194 | + |
| 195 | +- Chrome/Edge: Latest 2 versions |
| 196 | +- Firefox: Latest 2 versions |
| 197 | +- Safari: Latest 2 versions |
| 198 | + |
| 199 | +## Contributing |
| 200 | + |
| 201 | +This module follows Apache Hadoop's contribution guidelines. All source files must include the Apache License header. |
| 202 | + |
| 203 | +### Code Style |
| 204 | + |
| 205 | +- TypeScript strict mode enabled |
| 206 | +- ESLint for code quality |
| 207 | +- Prettier for code formatting |
| 208 | +- Conventional commit messages |
| 209 | + |
| 210 | +## Developer Documentation |
| 211 | + |
| 212 | +Detailed guides for extending and customizing the UI: |
| 213 | + |
| 214 | +- **[Design Document](src/main/webapp/docs/design_doc.md)** - Architecture, design decisions, and technical specifications |
| 215 | +- **[Adding Validation Rules](src/main/webapp/docs/development/adding-validation-rules.md)** - Guide for implementing custom validation rules |
| 216 | +- **[Extending Scheduler Properties](src/main/webapp/docs/development/extending-scheduler-properties.md)** - Instructions for adding new configuration properties |
| 217 | + |
| 218 | +## License |
| 219 | + |
| 220 | +Licensed under the Apache License, Version 2.0. See the LICENSE file in the Hadoop root directory. |
| 221 | + |
| 222 | +## Related Documentation |
| 223 | + |
| 224 | +- [YARN Capacity Scheduler Documentation](../hadoop-yarn-site/src/site/markdown/CapacityScheduler.md) |
| 225 | +- [YARN REST API Documentation](../hadoop-yarn-site/src/site/markdown/ResourceManagerRest.md) |
| 226 | +- [Hadoop Main Documentation](https://hadoop.apache.org/docs/current/) |
0 commit comments