EVG JS API

Embedded Vector Graphics JavaScript API. More...

Data Structures
interface	Canvas
interface	IRect
interface	Rect
interface	Point2D
interface	AlphaCallback
interface	Path
interface	PenSettings
interface	Matrix2D
interface	ColorMatrix
interface	Color
interface	Colorf
interface	Stencil
interface	Texture
interface	ConvolutionKernel
interface	Text
interface	TextMeasure
interface	Matrix
interface	Vec3f
interface	Rectf
interface	Vec4f
interface	VertexAttribInterpolator
interface	VertexAttrib
interface	BlitParameters
interface	Mesh

Enumerations
enum	ShaderType { GF_EVG_SHADER_FRAGMENT =1 , GF_EVG_SHADER_VERTEX }
enum	AttributeMapType { GF_EVG_VAI_VERTEX_INDEX =0 , GF_EVG_VAI_VERTEX , GF_EVG_VAI_PRIMITIVE }

Functions
void	push ()
long	push (DOMString left_val, DOMString operand, DOMString right_val, optional DOMString right_val2=null)
long	push (DOMString cond_val, DOMString left_val, DOMString operand, DOMString right_val)
long	push (DOMString end_cond_val)
long	push (DOMString goto_val, long stack_index)
long	push (DOMString goto_val, DOMString stack_index_uniform)
void	update ()
long	PixelSize (DOMString pixel_format)

Detailed Description

This module provides bindings for GPAC evg_grp engine called EVG (for Embedded Vector Graphics).

The API is loaded as a JS module with the name "evg":

import * as evg from 'evg'
 
...

The API supports all Vector Graphics from evg_grp, path_grp, bindings for 2D and 3D matrices and text rendering as used by the compose_grp.

The API is a low level access to the software rasterizer and does not follow the Canvas2D API (https://www.w3.org/TR/2dcontext/) but most of the tools required to implement Canvas2D are present in the evg API.

It allows a filter to:

• generate new video frames from scratch
• use input video frames to generate new frames
• generate vector graphics and text directly on input video frames

Whenever pixel coordinates are indicated, 0,0 indicates:

• the center of the surface if the surface uses centered coordinates
• the top-left of the surface if the surface does not use centered coordinates

Whenever using an object as a parameter to a function, only the parameter names as listed in the corresponding interface object are checked. The name of the interface is not, and it usually does not define any constructor.

Note

Although 10-bit raster is supported for YUV formats (framebuffer and textures), colors are internally handled as 8-bits for solid brush and gradients. This will be updated in the near future.

The 3D extension of Canvas object can be used to draw vectorial 3D meshes and 2D paths.

Warning

The 3D rasterizer is much much slower than WebGL as it is only software and not realy optimized, do not use it for complex 3D scenes.

The 3D rasterizer is still an experimental project: it might not be completely stable and its API may change a lot in the near future.

The 3D rasterizer can however be useful when drawing simple 3D objects over an existing video, as it avoids GPU transfers of the frame data.

The rasterizer supports:

• points, lines, triangles, quads, strips and fans
• 2D texturing using RGBA or YUVA colorspace
• perspective correct interpolation
• simple antialiasing
• depth buffer (float only)
• rasterizing in YUVA or RGBA space
• a simple shading language for both vertex and fragment processing

The rasterizer does not claim compliancy to any GPU specification, and will likely produce different results than WebGL.

---

Data Structure Documentation

IRect

interface IRect

rectangle object, pixel coordinate

Data Fields
unsigned long	x	x coordinate in pixels of top-left corner y coordinate in pixels of top-left corner
unsigned long	w	width of rectangle in pixels
unsigned long	h	height of rectangle in pixels

Rect

interface Rect

rectangle object

Data Fields
unsigned long	x	x coordinate of top-left corner y coordinate of top-left corner
unsigned long	w	width of rectangle
unsigned long	h	height of rectangle

Point2D

interface Point2D

point object

Data Fields
unsigned long	x	x coordinate of point y coordinate of point

PenSettings

interface PenSettings

Pen settings, see GF_PenSettings

Data Fields
attribute double	width	pen line width
attribute double	miter	The miter limit of the line joins
attribute double	offset	The initial dash offset in the outline. All points before this offset will be ignored when building the outline
attribute double	length	The author-specified path length. Ignored if <= 0
attribute unsigned long	cap	The style of the lines ends
attribute unsigned long	join	The style of the lines joins
attribute unsigned long	align	The alignment of the outline with regard to the path
attribute unsigned long	dash	The dash style of the line
attribute Array	dashes	The dash pattern used for custom dashing

Color

interface Color

color object

Data Fields
unsigned long	r	red component - if absent, default to 0
unsigned long	g	green component - if absent, default to 0
unsigned long	b	blue component - if absent, default to 0
unsigned long	a	alpha component - if absent, default to 255

Colorf

interface Colorf

color object

Data Fields
double	r	red component - if absent, default to 0
double	g	green component - if absent, default to 0
double	b	blue component - if absent, default to 0
double	a	alpha component - if absent, default to 1.0

ConvolutionKernel

interface ConvolutionKernel

convolution kernel interface

Data Fields
attribute unsigned long	w	kernel width - shall be an odd number
attribute unsigned long	h	kernel height - shall be an odd number
attribute unsigned long	norm	kernel normalization. If 0 or undefined, no normalization is applied
attribute Array	k	kernel coeficients. The coeficients are unsigned integers, and there shall be at least w * h coefficients

TextMeasure

interface TextMeasure

text measurement interface

Data Fields
double	width	measured width of text
double	height	measured height of text
unsigned long	em_size	size of EM box of font
unsigned long	ascent	ascent of font
unsigned long	descent	descent of font
unsigned long	line_spacing	line spacing of font
unsigned long	underlined	underline position in EM box
unsigned long	baseline	baseline position in EM box
unsigned long	max_advance_h	max horizontal advance of any glyphs in font
unsigned long	max_advance_v	max vertical advance of any glyphs in font
boolean	right_to_left	set to true if text is right to left, false otherwise

Vec3f

interface Vec3f

3D vector object

Data Fields
attribute float	x	X coordinate
attribute float	y	Y coordinate
attribute float	z	Z coordinate

Rectf

interface Rectf

rectangle object

Data Fields
attribute float	x	X coordinate
attribute float	y	Y coordinate
attribute float	width	width
attribute float	height	height

Vec4f

interface Vec4f

4D vector object

Data Fields
attribute float	x	X coordinate
attribute float	y	Y coordinate
attribute float	z	Z coordinate
attribute float	w	4th dimension same as 'q' or 'alpha'
attribute float	q	4th dimension same as 'w' or 'alpha'
attribute float	alpha	4th dimension same as 'q' or 'w'

BlitParameters

interface BlitParameters

Parameter object for canvas texture blit

Data Fields
DOMString	mode	name of the scale mode - see gpac -hx ffsws. If undefined, let FFMPEG decide
double	p1	first parameter of scale mode, if any
double	p2	second parameter of scale mode, if any

Enumeration Type Documentation

ShaderType

enum ShaderType

defined shader types

Enumerator
GF_EVG_SHADER_FRAGMENT	shader is a fragment shader
GF_EVG_SHADER_VERTEX	shader is a vertex

AttributeMapType

enum AttributeMapType

Map type for vertex attributes and vertex attribute interpolators

Enumerator
GF_EVG_VAI_VERTEX_INDEX	one vertex attribute is given for each vertex index
GF_EVG_VAI_VERTEX	one vertex attribute is given for each vertex
GF_EVG_VAI_PRIMITIVE	one vertex attribute is given for each primitive

Function Documentation

push() [1/6]

void push

(

)

Shader object common to vertex and fragment shaders

Although QuickJS is an amazing piece of software, calling JavaScript for each fragment is just too costly. The shader object provides a simple set of tools to produce a vertex or fragment output at reasonnable speed, without interacting with JavaScript.

Generic principles

The working principles of the shader object is as follows:

• no complex language processing (eg GLSL) and basic validation only (at least for the time being)
• instructions and variables are pushed onto a stack of instructions
• native EVG objects (Matrix, VertexAttrib, VertexAttribInterpolator, Texture) are stored as pointers to the native objects
• variables can be defined as uniforms, and updated whenever needed
• upon executing the shader, the instructions are processed in order

Local variables are untyped, except for reserved variables. The type is inferred whenever the variable is assigned or modified, thus a same variable can change type during the execution of the shader. Upon assigning, the type of the source variable is inferred as follows:

• Javascript bool and integer are converted to integer type
• Javascript float is converted to float type
• A javascript object with properties {x, y, z, q/w} or {r,g,b,a} is converted to a 4 component vector
• A javascript object with properties {x, y, z} or {r,g,b} is converted to a 3 component vector
• A javascript object with properties {x, y} or {r,g} or {s,t} is converted to a 2 component vector
• A javascript array is converted to a 1, 2, 3 or 4 float vector
• A Matrix object is kept as a pointer
• A VertexAttribInterpolator object is converted to a 2D, 3D or 4D vector depending on the number of components in the object
• A Texture sample is converted to a 4D vector

Note

There is no support for structures nor arrays in the shader instructions.

Warning

A local variable can only be of type bool, integer, float or vector of float (up to 4 dimension). It is not possible to store an EVG object (Matrix, VertexAttrib, VertexAttribInterpolator, Texture) as a local variable.

The assignment of vectors can use masking of components, using a combination of x, y, z, q (or r, g, b, a or s,t) values: \code shader.push('foo', '=', 'val.xy'); shader.push('foo.xy', '=', 'val.xz'); \endcode \warning there is no swizzling , .xy is equivalent to .yx`

Fragment shader reserved variables

These variables names are reserved in a fragment shader, and their type cannot be modified:

Variable name	type	Semantic
fragColor	Vec4f	the fragment output color in RGBA colorspace
fragRGBA	Vec4f	the fragment output color in RGBA colorspace
fragYUVA	Vec4f	the fragment output color in YUVA colorspace
fragX	float	the fragment X coordinate
fragY	float	the fragment Y coordinate
fragDepth	float	the fragment depth value - can be overwritten by shader
fragZ	float	the fragment Z coordinate
fragW	float	the fragment W coordinate
txCoord	float	the texture coordinates for 2D shaders
txCoordi	ints	the integer texture coordinates for 2D shaders
fragOdd	boolean	the odd/even flag of 2D path for 2D shaders

Note

txCoordi is used to fetch the pixel value as int. This is quite experimental, and should only be used to assign fragRGBA or fragYUVA.

Vertex shader reserved variables

These variables names are reserved in a vertex shader, and their type cannot be modified:

Variable name	type	Semantic
vertex	Vec4f	the input vertex coordinates
vertexOut	Vec4f	the output vertex coordinates

Uniforms variables

Uniforms variables are variables whose values need to be updated but do not require a change in the shader logic. Since some of these variables may be native JS interpreter types, they cannot be kept by reference in the stack. Instead, these variables are passed by names, and correspond to properties of the shader object. These properties are evaluated whenever the Shader.update function is called.

Uniforms are referenced in the shader as string variables prefixed with '.'; for example '.foo' will correspond to the 'foo' property of the shader object. The associated JS values can be:

• boolean
• integer
• float
• array of float, which is converted to Vec2f, Vec3f or Vec4f
• Float32Array or ArrayBuffer, which is converted to Vec2f, Vec3f or Vec4f

Example:

//shader creation part
...
shader.push('my_var', '=', '.somevar');
... do something with my_var
shader.push('fragRGBA', '=', 'myvar');
 
 
//setup draw
canvas.fragment = shader
shader.somevar = [0.5, 0.5, 0.5, 1.0];
shader.update();
//draw

Operations

Operations take zero, one or two right values and assign the result to a left value.

The left value can only be:

• a built-in variable or a local variable in a fragment shader.
• a built-in variable, a local variable or a VertexAttribInterpolator in a vertex shader.

The first right value can be a built-in variable, a local variable, a uniform, a Texture, a VertexAttrib (vertex shader only), a VertexAttribInterpolator (fragment shader only) or a Matrix.

Note

There is currently no support for matrix product in the shaders

Single-dimension operations on vectors are done component-wise, eg res.x = sin(rv1.x), res.y = sin(rv1.y), ...

Note

in the following table, rv1 means first right value, rv2 means second right value if present for the operation

Whenever rv2 is allowed, this can only be an already locally defined variable, and cannot be a uniform or a JS Object.

Whenever rv2 is allowed, this can only be an already locally defined variable, and cannot be a uniform or a JS Object.

Both the left value and the right value can be assigned a component mask:

Name	rv1	rv2	Semantic
'='	any	none	assigns rv1 to left value (1)
'*='	any	none	multiplies left value by rv1 (1)
'*='	Matrix	none	multiplies left value by the given matrix
'/='	any	none	divides left value by rv1 (1)
'+='	any	none	adds right value to rv1 (1)
'=!'	any	none	assigns not rv1 to left value (left = !right) (1)
'normalize'	Vec3f	none	assigns normalize value of right vector to left value
'length'	Vec3f	none	assigns length of right vector to left value (float type)
'distance'	Vec3f	Vec3f	assigns distance between the two right values to left value (float type)
'dot'	Vec3f	Vec3f	assigns dot product of rv1 by rv2 to left value (float type)
'cross'	Vec3f	Vec3f	assigns cross product of rv1 by rv2 to left value (Vec3f)
'pow'	any	same as rv1	assigns pow of rv1 by rv2 to left value
'sin'	any	none	assigns sin(rv1) to left value
'asin'	any	none	assigns asin(rv1) to left value
'cos'	any	none	assigns cos(rv1) to left value
'acos'	any	none	assigns acos(rv1) to left value
'tan'	any	none	assigns tan(rv1) to left value
'atan'	any	any	assigns atan(rv1, rv2) to left value
'log'	any	none	assigns log(rv1) to left value
'exp'	any	none	assigns exp(rv1) to left value
'log2'	any	none	assigns log2(rv1) to left value
'exp2'	any	none	assigns exp2(rv1) to left value
'sinh'	any	none	assigns sinh(rv1) to left value
'cosh'	any	none	assigns cosh(rv1) to left value
'sqrt'	any	none	assigns sqrt(rv1) to left value
'inversesqrt'	any	none	assigns 1/sqrt(rv1) to left value
'abs'	any	none	assigns absolute value (ABS) of rv1 to left value
'sign'	any	none	assigns sign vector of rv1 to left value - cf GLSL
'floor'	any	none	assigns floor(rv1) to left value
'ceil'	any	none	assigns ceil(rv1) to left value
'fract'	any	none	assigns fractional part of rv1 to left value - cf GLSL
'mod'	any	any	assigns modulo of rv1 by rv2 to left value - cf GLSL
'min'	any	any	assigns minimum of rv1 and rv2 to left value - cf GLSL
'max'	any	any	assigns maximum of rv1 and rv2 to left value - cf GLSL
'clamp'	any	any	clamps left value with rv1 as minimum and rv2 as maximum
'sampler'	Texture	Vec2f	sets left value to the pixel value of the texture rv1 at coordinate rv2, as an RGBA Vec4f
'samplerYUV'	Texture	Vec2f	sets left value to the pixel value of the texture rv1 at coordinate rv2, as a YUVA Vec4f
'discard'	none	none	discards the current fragment and exits shader processing (fragment shader only)
'print'	any	none	prints the given variable rv1 to stderr
'toRGB'	Vec3f, Vec4f	none	converts rv1 to RGB
'toYUV'	Vec3f, Vec4f	none	converts rv1 to YUV

(1): these operations accept component masks on both left value and right value. Example

shader.push('myval.xy', '*=' 'otherval.zq');

If the right value has less component indicated in the mask than left value components, the last indicated component of the right value will be used for the remaining left value components. Example

//myval will be assigned to {x:otherval.z, y:otherval.q, z:otherval.q}
shader.push('if', 'myval.xyz', '=' 'otherval.zq');

Conditions

Conditions are expressed with the following keywords:

• 'if' : takes 3 additional parameters left, operandand right and evaluates the condition
• 'elseif' : takes 3 additional parameters left, operand and right and evaluates the condition
• 'else': no additional parameters
• 'end': no additional parameters

Conditions can be nested.

Name	Right Value	Semantic
'<'	any	evaluate to true if left is strictly less than right
'<='	any	evaluate to true if left is less than or equal to right
'>'	any	evaluate to true if left is strictly greater than right
'>='	any	evaluate to true if left is greater than or equal to right
'=='	any	evaluate to true if left is equal to right
'!='	any	evaluate to true if left is different from right

Example:

...
shader.push('if', 'my_var.x', '<=', 0.5);
shader.push('if', 'my_var.y', '>=', 0.5);
...
shader.push('end');
shader.push('elseif', 'my_var.x', '<=', '.uniform');
...
shader.push('else');
...
shader.push('end');
...

Conditions accept component masks on both left value and right value, resulting in a per-component of the mask test. Example

shader.push('if', 'myval.xy', '<' 'otherval.zq');

If the right value has less component indicated in the mask than left value components, the last indicated component of the right value will be used for the remaining left value components. Example

//myval will be tested against {x:otherval.z, y:otherval.q, z:otherval.q}
shader.push('if', 'myval.xyz', '<' 'otherval.zq');

Stack jumping

The goto statement allows jumping in the stack. The goto statement takes a single parameter which can either be an integer or a uniform name resolving to an integer. This integer gives the 1-based index of the instruction to jump to. Example:

...
shader.label_start = shader.push('if', 'my_var.x', '<=', 0.5);
shader.push('if', 'my_var.y', '>=', 0.5);
...
shader.push('goto', '.label_end'); //go to the instruction at position label_end
...
shader.push('end');
shader.push('elseif', 'my_var.x', '<=', '.uniform');
shader.push('goto', '.label_start'); //go to the instruction at position label_start
shader.push('else');
shader.push('goto', 0); //go to beginning of the instruction stack
shader.label_end = shader.push('end');
...

The value of the jump position is not checked while assigning the goto instruction, it is checked while processing the fragment: this allows jumping anywhere in the stack.

Warning

You must make sure that your shader state is correct when jumping in the stack of instructions !

Note

There is no provision for while and for statements in the shader. This can however be emulated with the goto statement.

Vertex attribute interpolation

The rasterizer performs perspective-correct interpolation of attributes, and provides two objects, VertexAttrib and VertexAttribInterpolator, to handle interpolation. The attribute interpolation can be done:

• at the fragment shader stage only
• at both vertex and fragment shader stages

Fragment-shader only interpolation

In this configuration, the attributes to interpolate do not depend on the state of the vertex shader (eg for example vertices colors or texture coordinates). To interpolate the attribute, a VertexAttribInterpolator object is used to wrap the attributes to interpolate.

Example:

let txi = new evg.VertexAttribInterpolator(cube_txcoords, 2, GF_EVG_VAI_VERTEX);
frag_shader=canvas.new_shader(GF_EVG_SHADER_FRAGMENT);
 
frag_shader.push('txc', '=', txi);
frag_shader.push('txc', '*=', 2);
frag_shader.push('txval', 'sampler', texture, 'txc');
frag_shader.push('fragColor', '=', 'txval');

Per-vertex interpolation

In this configuration, the attributes to interpolate depend on the state of the vertex shader. To interpolate the attribute, a VertexAttrib object is used to wrap the attributes to interpolate at the vertex shader, and a VertexAttribInterpolator object is used to perform the interpolation at the fragment shader.

Example:

let ni = new evg.VertexAttribInterpolator(3);
ni.normalize=true;
let normals = new evg.VertexAttrib(cube_normals_face, 3, GF_EVG_VAI_PRIMITIVE);
 
vert_shader=canvas.new_shader(GF_EVG_SHADER_VERTEX);
vert_shader.push('normal', '=', normals);
vert_shader.push('normal', '*=', normal_matrix);
vert_shader.push(ni, '=', 'normal');
vert_shader.push('vertex', '*=', matrix);
vert_shader.push('vertexOut', '=', 'vertex');
 
frag_shader=canvas.new_shader(GF_EVG_SHADER_FRAGMENT);
 
frag_shader.push('normal', '=', ni);
frag_shader.push('normal.q', '=', 1.0);
frag_shader.push('fragColor', '=', 'normal');

RGB and YUV color spaces

The rasterizer can work on both RGB and YUV output color buffers.

• When a fragment output value is given in RGB and the output color buffer is in YUV, conversion is performed
• When a fragment output value is given in YUV and the output color buffer is in RGB, conversion is performed
• Otherwise, no conversion is done

Texture sampling can also work on both RGB and YUV texture maps.

• When a sampler call is made on a YUV texture, conversion is performed
• When a samplerYUV call is made on a RGB texture, conversion is performed
• Otherwise, no conversion is done

The toRGB and toYUV operations can also be used to convert between color space, but precomputed values should be preferred.

*/ interface Shader { /*! resets the stack of instructions

push() [2/6]

long push	(	DOMString	left_val,
		DOMString	operand,
		DOMString	right_val,
		optional DOMString	right_val2 = null )

pushes a new instruction on the stack

Parameters

left_val	the name of the left value to be assigned or modified
operand	the name of the operand to use
right_val	the name of the right value to use by the operand
right_val2	the name of the second right value to use by the operand, if any

Returns

the 1-based index of the added instruction

push() [3/6]

long push	(	DOMString	cond_val,
		DOMString	left_val,
		DOMString	operand,
		DOMString	right_val )

pushes a new condition instruction on the stack

Parameters

cond_val	the name of the condition. Allowed values are if, elseif
left_val	the name of the left value to be assigned or modified
operand	the name of the operand to use
right_val	the name of the right value to use by the operand

Returns

the 1-based index of the added instruction

push() [4/6]

long push

(

DOMString

end_cond_val

)

pushes an end of condition instruction on the stack

Parameters

end_cond_val

the name of the end of condition. Allowed values are else, end

Returns

the 1-based index of the added instruction

push() [5/6]

long push	(	DOMString	goto_val,
		long	stack_index )

pushes a goto instruction on the stack

Parameters

goto_val	the name of the goto. Allowed values are goto
stack_index	the 1-based index of the stack instruction to go to

Returns

the 1-based index of the added instruction

push() [6/6]

long push	(	DOMString	goto_val,
		DOMString	stack_index_uniform )

pushes a goto instruction on the stack

Parameters

goto_val	the name of the goto. Allowed values are goto
stack_index_uniform	name if the uniform giving the 1-based index of the stack instruction to go to

Returns

the 1-based index of the added instruction

update()

void update

(

)

updates uniforms in the shader

PixelSize()

long PixelSize

(

DOMString

pixel_format

)

queries size in bytes of a pixel for a given pixel format

Parameters

pixel_format

the desired pixel format

Returns

the size in bytes