CURRENT STATE OF WEBGL / GLSL

If you are paying attention close enough, you might have noticed that in the background of this blog is a very simple and strightforward wave simulation using GLSL or WebGL. It has specifically been written in accordance with the OpenGL Shading Language, Second Edition [1,2] text in mind so it is compatible with every single WebGL implementation ever conceived, and is almost completely decoupled from the browser. It has some intentional 'glitch' in it, which is a reference to the analog days of Sutherland's work.

Specifically, the simulation uses a textbook implementation of GLSL, as follows:

• GLSL 1.2
• OpenGL ES 2.0
• WebGL 1.0

The only coupling to the browser is the opening of a GL context, and if one clicks on the animation in the right place, an "un-project" operation that unwinds the Z-division takes place so that the fragment that underlies the mouse cursor can be calculated and the scene can be rotated using a very primitive rotation scheme which includes gimble lock (no quarternions here!). Both are extrordinarilly simple 3D graphics operations that should not affect rendering at all and is the absolute minimum level of coupling one might expect. In short, it is the perfect test of the most basic of WebGL capability.

The un-project operation is written with the minimal amount of code required and uses a little linear algebra trick to do it very efficiently. Feel free to inspect the code to see how it's done.

Current State

Update 26-Jun-23: After much trial and error, and testing on many many devices, I now have successfully isolated three separate WebGL bugs. Now that I have the three bugs properly isolated, I'm starting the writeup and hope to submit the following bug reports in the next week or two, as follows:

Incorrect rendering of WebGL 1.0 scene in Google Chrome
WebGL rendering heisenbug causes GL context to crash in Chrome after handling some N fragments
WebGL rendering heisenbug causes GL context to incorrectly render scene after handling some N fragments

TBC...

The current state as at 14-March-2023 is that Chrome and other browsers are not able to run this animation for more than 24 hours without crashing, and on the latest versions of Chrome released early March, the animation has now slowed down to ridiculous FPS levels. Previously the animation ran at well over 30 FPS on most devices, but would crash after 24 hours.

This animation will quite happily run on an old iPad running Safari, however Chrome currently seems to be struggling. The number of vertices and the the number of sin() operations that it needs to calculate is well within the capabilities of all modern processors, including those found in i-devices such as phones and tablets on which one can play a typical game.

Example of correct rendering on all modern browsers including Safari on iPad

Example of incorrect rendering on Chrome 111.0.5563.111 (64-bit) as at 23-Mar-23

Brave Browser (based on Chromium) renders correctly, but is horrendously slow in the GA branch (Beta currently works fine). I'm not sure if this is a Linux vs. Windows issue or discrete vs. embedded GPU at this stage, will investigate further when I have time.

NB. As at 14-March-2023, on Brave Browser 1.49.120 on Windows with a Discrete GPU the simulation struggles to render 5 FPS, and on Brave Browser 1.50.85 (beta) on Linux with an embedded GPU it works OK, but I can point to other vizualisation artefacts elsewhere that 1.50.85 cannot handle (but which previous versions of Brave/Chrome could), for example, on the homepage of vizdynamics.com, the Humanized Data Robot should gently move around the screen with a slight parallax effect if one moves the mouse over it. Why is the rendering engine in Chrome suddenly regressing, and why is it not using the GPU? This wave simulation should be able to run in it's entirety on SIMD architecture and the Humanized Data Robot used to be rendering flawlessly. What is going on?

At 30 FPS, the wave simulation requires around 75 MFLOPS of processing power. To put that into perspective, the first Sun Microsystems SPARC Station released in 1989 was able to calculate 16.2 MIPS (similar to MFLOPS), and the SPARC Station 2 (released 1991) could calculate nearly 30 MIPS. That was over 30 years ago, and a SPARC Station 2 machine had enough compute power that it could happily calculate the same wave simulation at around 10-15 FPS without vizualising it, but actually at 2-5 FPS once one implements a GPU pipeline (thank god SGI bought out IRIS GL).

I still have my copy of the original OpenGL Programming Guide (1st edition, 6th printing) that came with my Silicon Graphics Indy workstation. It was a curious book, and I implemented my first OpenGL version of these wave simulations in 1996 according to it, so I'm quite familiar with what to expect. An Indy could handle this - with a bit of careful tuning - quite well. The hardness of this vizualisation is the tremendous number of sin() operations, and the cosine()s used to take their derivative, so it really does test the compute power of a graphics pipeline quite well - if the machine or the implementation isn't up to it, these calculations will bring it to it's knees quite quickly.

Fast-forward to 2023, and a basic i7 cannot run the simulation at 30FPS! To put things into perspective, a 2010 era Intel i7 980 XE is capable of over 100 GFLOPS (about 1000x more processing power than whats required to do 75 MFLOPS), and that's without engaging any discrete or integrated SIMD GPU cores. Simply put, the animation in the background of this blog should be trivial for any computing device available today, and should run without interruption.

Lets see how well things progress through March and if things improve.

Update 30-Jan-24: Adding an FFT to make the wave simulation faster makes the problem go away, or potentially causes it to take longer to crash. Dunno. Have asked the Brave team to look into it:

Hey @brave @BraveSupport @BrendanEich While I have your attention:

I have isolated the long-standing bug with long running WebGL memory leaking till the browser crashes in Chromium on some GPUs. #SIGGRAPH @siggraph

More info here - please submit to the Chromium team, I…

— AP on CompSciFutures (∀/acc) (@CompSciFutures) January 29, 2024

References

[1] Rost, Randi J., and John M. Kessenich. OpenGL Shading Language. 2nd ed. Addison-Wesley, 2006. OpenGL 2.0 + GLSL 1.10

[2] Shreiner, Dave. OpenGL Programming Guide the Official Guide to Learning OpenGL, Version 2. 5th ed. Upper Saddle River, NJ u.a: Addison-Wesley, 2006.

YOU ARE ALL IN A VIRTUAL

I love Urban Dictionary - watch out for my @CompSciFutures updates. Here's one I posted today:

This is actually a commentary on attack surfaces, specifically that back in the day, one's lead architect knew the entirety of a system's attack surface and would secure it appropriately. Today, systems are so complex, that no single man knows the entire intricacies of the attack surface for even just a small web application with an accompanying mobile app. The security implications of this are profound, hence the reason why I am writing a textbook on the topic.

Link to more Urban Dictionary posts in the footer.

SOFTWARE ENGINEERING MANUAL OF STYLE, 3RD EDITION

My apologies to everyone I was supposed to follow up with in January - I've been writing a textbook. I'll get back to you late February/early March, I'm locking down and getting this done so we can address the systemic roots of this ridicuous cyber security problem we have all found ourselves in.

The book is called:

The Software Engineering Manual of Style, 3rd Edition
A secure by design, secure by default perspective for technical and business stakeholders alike.

The textbook is 120 pages of expansion on a coding style guide I have maintained for over 20 years and which I hand to every engineer I manage. The previous version was about 25 pages, so this edition is a bit of a jump!

Secure-by-design, secure-by-default software engineering. The handbook.

It covers the entirety of software engineering at a very high level, but has intricate details of information security baked into it, including how and why things should be done a certain way to avoid building insecure technology that is vulnerable to attack. Not just tactical things, like avoiding 50% of buffer overruns or most SQL injection attacks (and leaving the rest of the input validation attacks unaddressed). This textbook redefines the entire process of software engineering, from start to finish, with security in mind from page 1 to page 120.

Safe coding and secure programming are not enough to save the world. We need to start building technology according to a secure-by-design, secure-by-default software engineering approach, and the world needs a good reference manual on what that is.

This forthcoming textbook is it.

Latest Excerpts

21-Feb-23 Excerpt: The Updated V-Model of Software Testing
(DOI: 10.13140/RG.2.2.23515.03368)

21-Feb-23 Excerpt: The Software Engineering Standard Model
(DOI: 10.13140/RG.2.2.23515.03368)

EDIT 22-Mar-23: Proof showing that usability testing is no longer considered non-functional testing
(DOI: 10.13140/RG.2.2.23515.03368)

EDIT 22-Mar-23: The Pillars of Information Security, The attack surface kill-switch riddle +
The elements of authenticity & authentication (DOI: 10.13140/RG.2.2.12609.84321)

EDIT 22-Apr-23: The revised Iterative Process of Modelling & Decision Making
(DOI: 10.13140/RG.2.2.11228.67207/1)

EDIT 18-May-23: The Lifecycle of a Vulnerability
(DOI: 10.13140/RG.2.2.23428.50561)

Audience

I'm trying to write it so it's processes and methodologies:

• Can be baked into a firm by CXOs using strategic management principles; or
• embraced directly by engineers and their team leaders without the CEOs shiny teeth and meddlesome hands getting involved.

Writing about very technical matters for both audiences is hard and time consuming, but I think I'm getting the hang of it!

Abstract from the cover page

The foreword/abstract from the first page of the text reads as follows:

"The audience of this textbook is engineering based, degree qualified computer science professionals looking to perfect their art and standardise their methodologies and the business stakeholders that manage them. This book is not a guide from which to learn software engineering, but rather, offers best practices canonical guidance to existing software engineers and computer scientists on how to exercise their expertise and training with integrity, class and style. This text covers a vast array of topics at a very high level, from coding & ethical standards, to machine learning, software engineering and most importantly information security best practices.

It also provides basic MBA-level introductory material relating to business matters, such as line, traffic & strategic management, as well as advice on how to handle estimation, financial statements, budgeting, forecasting, cost recovery and GRC assessments.

Should a reader find any of the topics in this text of interest, they are encouraged to investigate them further by consulting the relevant literature. References have been carefully curated, and specific sections are cited where possible."

The book is looking pretty good: it is thus far what it is advertised to be.

Helping out and donating

The following link will take you to a LinkedIn article that I am publishing various pre-print extracts (some are also published above).

If you are in the field of computer science or software engineering, you might be able to help by providing some peer-review. If not, there is a link to an Amazon booklist that you can also contribute to this piece of work by donating a book or two.

Visit the book's homepage on LinkedIn »

And feel free just to take a look and see where we're going and what's being done to ensure that moving forward, we stop engineering such terribly insecure software. Any support to that end would be most appreciated.

Edited 22-Mar-23: added usability testing proof
Edited 22-Mar-23: added Pillars of Cybersecurity

VIZLAB 2.0 CLOSURE

VizDynamics is still a trading entity, but VizLab 2.0 is now closed. The lack of attention to cybersecurity by state actors, big tech and cloud operators globally made it impossibly difficult to continue operating an advanced computer science lab with 6 of Melbourne's best computer scientists supporting corporate Australia. We could have continued, but we saw this perfect storm of cyber security coming and decided to dial down VizLab starting in 2017. Given recent cyber disasters, it is clear we made the right decision.

State actors, cloud operators and big tech need to be careful with “vendor backdoor” legislation such as key escrow of encryption, because this form of 'friendly fire' hides the initial attack vector and the intial point of network contact when trying to forensically analyse and close down real attacks. Whilst that sort of legislation is in place without apporpriate access controls, audit controls, detective controls, cross-border controls, kill switches and transparency reporting, it is not commercially viable for us to operate a high powered CS lab, because all the use cases corporate Australia want us to solve involve cloud-based PII, for example, ‘Prediction Lakes’.

A CS Lab designed for paired programming

Part MIT Media lab, part CMU Autonlab, part vizualistion lab, part paired-programming heaven: this was VizLab 2.0.

VizLab doorway with ingress & egress Sipher readers, Inner Range high frequency monitoring & enterprise class CCTV.
A secure site in a secure site.

A BSOD in The Lab: The struggles of WebGL and 3D everywhere.
Note the 3-screen 4k workstation in the foreground designed
for paired local + remote programming.

The Lab - Part vizualisation, part data immersion. VizLab 2.0.
Note the Eames at the end of the centre aisle.

Each workstation was setup for paired programming. 6 workstations, where you could plug in 2 keyboards, 2 mice, 2 chairs side-by-side and enough room so that you weren't breathing on eachother with returns either side big enough for a plethora of academic texts, client specs and all the notes you could want. With 2 HDMI cables hung from the roof linking to projectors on opposing walls that could reach any workstation at any moment, this was an environment for working on hard things; collaboratively, together.

Note the lack of client seating. They would have to perch on the edge of a desk and see everything, or an Eames lounge at the end of the room and try to see everything, or a White real leather Space Furniture couch next to a Tom Dixon and see nothing - we took host to management teams from a plurality of ASX200s, and the first thing that struck them — we didn't have seating for them, because the next few hours they were going to be moving around and staring at walls, computers, the ceiling — there simply was nowhere to sit, when a team of computer scientists, trained in computer graphics and very deft with data science were taking them on a journey into Data.

The now: VizLab 3.0 – The future: VizLab 4.0

AP is still around and is spending the most part of 2023 writing a textbook on secure software engineering, and we've setup a smaller two-man VizLab 3.0 for cyber defence research mainly around GRC assessment and computer science education both at a secondary and a tertiary level. AP is currently doing research in that field so we can hopefully reduce cyber-risk down to a level that is acceptable to coprorate Australia by increasing the mathematical and cyber-security awareness and literacy of computer science students as they enter into university and then industry.

If we can help to create that environment, then VizLab 4.0 may materialise and will be bigger and better, but because we dialed back our insurances (it's not practical to be paying $25K pa while we're doing research), we aren't in a position to provide direct consultation services at the moment. VizDynamics is still a trading entity, and a new visualisation based Information Security brand might be launched somewhere in 2024 based on the “Humanizing Data” vizualisation thesis (perhaps through academia or government – we’re not sure yet). Fixes are happening to WebGL rendering engines, and slowly cyber security awareness is rising to the top of the agenda, so our work is slowly shifting the needing and we’re moving in the right direction.

If you want to keep track of what AP's up to, vizit blog.andrewprendergast.com.