Zen User Intereactions

I envision that there will be various levels and classes of Zen user interaction. I will try to list them in this wiki page.

Zen user interactions based upon hooking up HTML5 APIs to Zen's Scheme-in-the-browser

Note that the full list of HTML5 APIs is quite long, but below I've listed some that might interest us the most. I have left out APIs that don't work on the popular web browsers (except for Explorer, which I ignore) and APIs that don't work on Safari iOS (with a view that Zen might someday soon work on touch screens).

1. Pointer events - see note [1] below

2. Pointer Lock - see note [2] below

3. HTML drag and drop

4. File drag and drop

5. Clipboard

6. Intersection Observer API - well supported in browsers; see note [3] below

7. Navigation Timing API

8. Gamepad - not just for Xbox or PS, see MacWorld article or CNet article

9. Fetch - a bit like jQuery.ajax(), but better

10. Screen Capture API - well supported for desktop browsers

11. Media Source API

12. MediaStream Recording API

13. Canvas

14. WebGL

15. Channel messaging

16. Contact picker

17. Long Tasks API - not well supported by browsers, but might be useful for debugging

18. MediaStream Image Capture API - experimental, not well supported by browsers

19. Content index - experimental, not well supported by browsers

20. IndexedDB API - complicated API for Web Workers

21. Encrypted media

22. File and directory entries - don't use this until it is no longer marked experimental

23. File System Access - browser compatibility unknown, but this page's "See also" link leads us to a demo that works on Chrome (see the section "Try it")

24. WebSockets - two-way interactive communication session between the user's browser and a server

25. WebRTC API - interesting

Notes

1. Much of today's web content assumes the user's pointing device will be a mouse. However, since many devices support other types of pointing input devices, such as pen/stylus and touch surfaces, extensions to the existing pointing device event models are needed. Pointer events address that need.

2. The Pointer Lock API (formerly called Mouse Lock API) provides input methods based on the movement of the mouse over time (i.e., deltas), not just the absolute position of the mouse cursor in the viewport. It gives you access to raw mouse movement, locks the target of mouse events to a single element, eliminates limits on how far mouse movement can go in a single direction, and removes the cursor from view. It is ideal for first person 3D games, for example. More than that, the API is useful for any applications that require significant mouse input to control movements, rotate objects, and change entries, for example allowing users to control the viewing angle by moving the mouse around without any button clicking. The buttons are then freed up for other actions. Other examples include apps for viewing maps or satellite imagery.

3. The Intersection Observer API provides a way to asynchronously observe changes in the intersection of a target element with an ancestor element or with a top-level document's viewport. Historically, detecting visibility of an element, or the relative visibility of two elements in relation to each other, has been a difficult task for which solutions have been unreliable and prone to causing the browser and the sites the user is accessing to become sluggish. As the web has matured, the need for this kind of information has grown. Intersection information is needed for many reasons, such as... .

Zen's Adaptations of Raw HTML APIs

Sometimes it will be helpful either (1) to interact intimately with low-level HTML APIs like JavaScript.info's "Drag'n'Drop with mouse events" does or (2) to create a synthetic event to force the urgent handling of a situation that has built up through a series of low priority native events. Such adaptations of "raw" HTML APIs will explored in this section. An example of Zen's use of a synthetic event might be task #488 described in Zen's PivotalTracker project.

[To be developed]