Skip to content

Commit 39b50ec

Browse files
davepagurekdavepagurek
authored
Merge pull request #6442 from AryanKoundal/gsocreport
Work Product Report | Improving p5.js WebGL/3d functionality
2 parents cfe48f5 + 6c3a07d commit 39b50ec

File tree

2 files changed

+150
-0
lines changed

2 files changed

+150
-0
lines changed

‎contributor_docs/project_wrapups/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ This folder contains wrap-up reports from p5.js related [Google Summer of Code](
66
## Google Summer of Code
77

88
### Google Summer of Code 2023
9+
* [Improving p5.js WebGL/3d functionality(On-going)](https://github.com/processing/p5.js/blob/main/contributor_docs/project_wrapups/aryan_koundal_gsoc_2023.md) by Aryan Koundal, 2023
910
* [Mobile/Responsive Design Implementation of p5.js Web Editor](https://github.com/processing/p5.js/blob/main/contributor_docs/project_wrapups/dewansDT_gsoc_2023.md) by Dewansh Thakur, 2023
1011
* [Friendly Error System(FES) and Documentation](https://github.com/processing/p5.js/blob/main/contributor_docs/project_wrapups/ayush23dash_gsoc_2023.md) by Ayush Shankar, 2023
1112
* [Supporting shader-based filters in p5js](https://github.com/processing/p5.js/blob/main/contributor_docs/project_wrapups/wong_gsoc_2023.md) by Justin Wong, 2023

‎contributor_docs/project_wrapups/aryan_koundal_gsoc_2023.md‎

Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
# Improving p5.js WebGL/3d functionality
2+
3+
#### By Aryan Koundal(@AryanKoundal)
4+
#### Mentored by : Dave Pagurek(@davepagurek ), Tanvi Kumar(@TanviKumar )
5+
6+
## Overview
7+
8+
In p5.js, there are two render modes: P2D (default renderer) and WebGL. WEBGL
9+
enables the user to draw in 3D. There are many ways to implement lighting.
10+
Currently, p5js has implemented 8. To add lighting to a 3D object, one can
11+
use these functionalities. But there is another technique to light objects,
12+
not by direct light, but using the surrounding environment as a single light
13+
source, which we call Image-Based lighting.
14+
This project aims to add lighting to a 3D object, using the surrounding
15+
environment as a single light source, which is generally called Image-Based
16+
lighting. In simple words, one can very quickly drop in an image from real
17+
life to use as surrounding lights, rather than continuously tweaking the colors
18+
and positions of ambient, point, etc lights.
19+
20+
1. Diffused IBL
21+
1. Create scene's irradiance.
22+
1. Calculation of Scene’s irradiance for any direction.
23+
1. Sample the scene’s radiance from all possible directions, within the
24+
hemisphere, for each fragment.
25+
1. Take a finite number of samples of the scene's irradiance and pre-compute them.
26+
1. Create a pre-computed irradiance map.
27+
1. Sample the pre-computed irradiance map to retrieve diffuse irradiance.
28+
1. Specular IBL
29+
1. We need to preconvolute it.
30+
1. Use split sum approximation to divide the specular part into 2 further parts
31+
1. First one is a prefiltered environment map.
32+
1. Finalize Pre filter convolution using low discrepancy hammersley sequence
33+
and sample generation defined.
34+
1. Capturing prefilter mipmap levels, storing the results in prefiltered
35+
environment cubemap
36+
1. Second one is BDRF
37+
1. pre -calculate the BRDF’s response given input as roughness and angle
38+
between n and wi.
39+
Precalculations done using BRDF’S geometry function and Fresnel-Schlicks approximation.
40+
1. Stored in 2D lookup texture known as BRDF integration map.
41+
1. Combine the prefiltered environment map and BRDF integration map
42+
1. Combine the diffused and specular parts
43+
44+
45+
## Current State of The Project
46+
47+
The project is completed and here are some screenshots and videos demonstrating the work.
48+
49+
#### Image of Example 1
50+
![example 1](https://github.com/processing/p5.js/assets/77334487/8d818df0-17a8-4332-b369-bcb79a5afc67)
51+
52+
#### Video of Example 1
53+
https://github.com/processing/p5.js/assets/77334487/44b30c77-33c1-41d0-ada5-282424978832
54+
55+
#### Image oF Example 2
56+
![example 2](https://github.com/processing/p5.js/assets/77334487/e46f24b8-2713-4d2b-8392-744585da6a5b)
57+
58+
#### Video of Example 2
59+
https://github.com/processing/p5.js/assets/77334487/a0a6b3f9-b25b-451f-961e-b2970cb9e907
60+
61+
## Pull Request
62+
63+
- Pull Request: https://github.com/processing/p5.js/pull/6255
64+
- Work Reports:
65+
- [p5js Week 1 Report | GSoC’23 Processing Foundation](https://aryankoundal.medium.com/p5js-week-1-report-gsoc23-processing-foundation-9910934112e5)
66+
- [p5js Week 2 Report | GSoC’23 Processing Foundation](https://aryankoundal.medium.com/p5js-week-2-report-gsoc23-processing-foundation-c8a36f5cf34)
67+
- [p5js Week 3 Report | GSoC’23 Processing Foundation](https://aryankoundal.medium.com/p5js-week-3-report-gsoc23-processing-foundation-39043d0363e2)
68+
- and more...
69+
70+
71+
## Work Done
72+
73+
### src/webgl/light.js
74+
1. This includes the function `imageLight()` which provides the whole functionality.
75+
1. 2 Examples for the `imageLight()` are also included in this files.
76+
77+
### src/webgl/p5.RendererGL.js
78+
1. This includes 2 new maps, which are diffusedTextures and specularTextures for storing the p5 graphics.
79+
1. Then the function `getDiffusedTexture()` which gets called from _setImageLightUniforms. It's function is to create a blurry
80+
image from the input non blurry img, if it doesn't already exist.
81+
1. Also the function `getSpecularTexture()` which also gets called from _setImageLightUniforms. It's function is too create a texture
82+
from the input non blurry image, if it doesn't already exist
83+
1. Then the function `_setImageLightUniforms()`. It creates the textures and sets those textures in the shader being processed.
84+
85+
### src/webgl/p5.Texture.js
86+
1. This includes the `MipmapTexture` class.
87+
88+
89+
### src/webgl/shaders/imageLight.vert
90+
1. It is a vertex shader only for the image Light feature.
91+
92+
### src/webgl/shaders/imageLightDiffused.frag
93+
1. It is the Fragment shader for the calculations of the Diffused Part of Image Light.
94+
95+
### src/webgl/shaders/imageLightSpecular.frag
96+
1. It is the Fragment shader for the calculations of the Specular Part of Image Light.
97+
1. It has the functions `HammersleyNoBitOps()` , `VanDerCorput()`and `ImportanceSampleGGX()` which are
98+
referenced from "https://learnopengl.com/PBR/IBL/Specular-IBL"
99+
100+
### src/webgl/shaders/lighting.glsl
101+
1. This includes `calculateImageDiffuse()` and `calculateImageSpecular()` which actually calculates the output textures. These are calculated only when `imageLight()` is called.
102+
103+
### List of All shaders modified to improve webGL compatibility to newer versions
104+
1. src/webgl/shaders/basic.frag
105+
1. src/webgl/shaders/immediate.vert
106+
1. src/webgl/shaders/light.vert
107+
1. src/webgl/shaders/light_texture.frag
108+
1. src/webgl/shaders/line.frag
109+
1. src/webgl/shaders/line.vert
110+
1. src/webgl/shaders/normal.frag
111+
1. src/webgl/shaders/normal.vert
112+
1. src/webgl/shaders/phong.frag
113+
1. src/webgl/shaders/phong.vert
114+
1. src/webgl/shaders/point.frag
115+
1. src/webgl/shaders/point.vert
116+
1. src/webgl/shaders/vertexColor.frag
117+
1. src/webgl/shaders/vertexColor.vert
118+
119+
### Sketches I made, which might be useful
120+
1. Example 1 in imagelight https://editor.p5js.org/aryan_koundal/sketches/OEsyk6uZI
121+
1. Example 2 in imagelight https://editor.p5js.org/aryan_koundal/sketches/XjalPP7s4
122+
1. Final Prototype Diffused + Roughness (Variable Roughness) https://editor.p5js.org/aryan_koundal/sketches/4V790dB9Z
123+
1. Diffuse light final prototype https://editor.p5js.org/aryan_koundal/sketches/q_Zg37OB2
124+
1. Mipmap Specular roughness Prototype (visible variation) https://editor.p5js.org/aryan_koundal/sketches/Bi2BN7zjK
125+
1. Specular prefilterconvolution with chessboard https://editor.p5js.org/aryan_koundal/sketches/qaIhxRZHI
126+
1. Prefilterconvolution cubemaps prototypes 8 levels https://editor.p5js.org/aryan_koundal/sketches/YJTSFKhqt
127+
1. Specular prefilterconvolution with image https://editor.p5js.org/aryan_koundal/sketches/8divJgdQxO
128+
1. Specular mipmaps prototype with colors https://editor.p5js.org/aryan_koundal/sketches/3V9iDH8ax
129+
1. Sphere with point Lights https://editor.p5js.org/aryan_koundal/sketches/9NdeCtfdW
130+
1. Diffused texture Shader Prototype https://editor.p5js.org/aryan_koundal/sketches/DcFDcOFUn
131+
1. Resolution Pixelwise https://editor.p5js.org/aryan_koundal/sketches/5RfDIy2I9
132+
1. Cube reflecting mountain spheremap https://editor.p5js.org/aryan_koundal/sketches/2QS5-Fy0V
133+
1. Cube reflecting sky spheremap https://editor.p5js.org/aryan_koundal/sketches/O1NwI4ufp
134+
135+
## Further Improvement
136+
While working on this project, I realised that there is scope for improvement. Like increasing efficiency by using cubemaps instead of environment maps for Diffused Lighting. Also using framebuffers would improve the efficiency and reduce the time taken to render the lights.
137+
138+
## Conclusion
139+
140+
My GSoC experience was genuinely transformative, leading to significant personal and professional
141+
development within the open-source domain. I successfully overcame initial obstacles related to
142+
setting up the project and navigating the codebase, steadily advancing through my contributions.
143+
Throughout this summer, I actively participated in the p5.js open-source community, surpassing my
144+
initial expectations for my first major open-source venture and further igniting my enthusiasm. I
145+
want to express my heartfelt appreciation to my mentors, Dave Pagurek(@davepagurek ), and
146+
Tanvi Kumar(@TanviKumar ), for their invaluable guidance in navigating coding challenges.
147+
Effective teamwork, collaboration, and communication proved essential in this open-source journey.
148+
I eagerly anticipate expanding my contributions and honing my skills. This summer has instilled
149+
a sense of purpose in me, and I'm profoundly grateful for the mentorship and knowledge gained.

0 commit comments

Comments
 (0)