VKL - Videokernel Library

Table of Contents

• Overview
• Directory contents
• Linux (gcc on SL3)
• Root v4 on Linux
• Windows (MS Visual C++ v6)
• API reference

Overview

The video kernel library was designed for users of the PITZ Video System in order to be able to use the core functions of it. Included functionality is taking frames using socket connection, loading and saving of own image file formats (IMM, IMC, BKG, BKC) and applying certain algorithms, like creating average and envelope background, calculating significance mask and applying x-ray filtering.

The library was written in such a way that it is instantaneously possible to use it on Linux32, Linux64, Win32 and ROOT (32 and 64 bit) platforms. It may work on any other Little-Endian based platform with little adjustments. Big Endian (e.g. Sun) platform support is not done. This may change as it is requested by user demands.

To not be confused by the terminology, one should know that the term library in this document means a set of source code and header files to be used on many platforms. One may create by hand a shared library (.so) for Scientific Linux 3 (SL3) or a shared library (.dll) for Windows operating system using Visual C++ v6. These steps are not provided. The examples for Linux and Windows show how to integrate the main source and include files into example projects. Cross-platform compatibility is met by definition of data types (to be independent of 32 / 64 bit compilers) and by encapsulating certain functions inside platform.c and platform.h. For ROOT v4/v5 a makefile in order to create a shared library for using the functions inside ROOT C interpreter (CINT) is provided as example.

The library is expected to compile and link without any error on Linux, ROOT (v4, v5) and Windows. It was tested with Visual C++ V6 SP6, GCC on Scientific Linux 3 (SL3) and using ROOT v4.02.00 under Linux (SL3). The flawless execution on a 64-bit platform was tested under Linux.

Note: The library is only compatible with platforms that use Little-Endian byte ordering, like Intel X86 processors. Sourcecode has to be changed in certain positions if the library is going to be used on Solaris (e.g. using SPARC CPUs with Big-Endian byte ordering).

Directory Contents

Linux (gcc on SL3)

Using the supplied makefile getsequence.mak (make -f getsequence.mak) one should be able to compile the example code without errors or warnings on SL3. The result will be an executable file getsequence which should be working.
The example code inside getsequence.c is shown for reference and may not work (just because the test server to acquire frames from is not running by accident). The second makefile (getsequenced.mak) creates an executable getsequenceD with debugging information. This code was tested and proved to be working on 64-bit Linux. There are other makefiles (loadexample.mak, loadexampleD.mak) that create an example of loading and saving files. It is a good thing to do "make -f ... clean" first in order to purge object leftovers from earlier sessions.

Root v4/v5 on Linux

Using the supplied makefile rootlib.mak (make -f rootlib.mak) one should be able to compile the example root library without errors or warnings on SL3. It is a good idea to clean the current directory first (make -f rootlib.mak clean). It might be possible that the makefile needs to be adjusted (see commented-out ROOTSYS line and PLATFORM line) if the ROOTSYS variable is missing in the environment or points to a wrong installation. The second makefile (rootlibd.mak) creates a library (videokernelD.sl) with debug info. This code was tested and proved to be working on ROOT_64 under 64-bit Linux. The result of making will be a root shared library file videokernel.sl which can be loaded into Root interpreter and tested by using the following sequence:

	$> root
	
	root [0] gSystem->Load("videokernel.sl");
	root [1] .X rootexample.CXX
	root [2] .X rootexample_vfl.CXX
	

Note: The example code inside rootexample.CXX is shown for reference and may not work (just because the test server to acquire frames from is not running by accident). For rootexample_vfl.CXX the proper input file (see source) must be located at the appropriate directory.

Windows (MS Visual C++ v6)

After starting MS Visual C++ v6, one should load in the workspace file "getsequence.dsw". All pathes should be relative so directly compiling and linking without any warning or error should be no problem. The library and the provided example code was written to compile and link without a single error even at highest warning level ! If this does not work, one may check the include pathes as well as the output pathes for correctness. It is also a good advise to check the supplied files for read-only attribute and clear it in case it is set. The result will be an executable file getsequence.exe which should be working.
Another project workspace was added recently. It is called "vfload.dsw" and show loading/saving of files using the video kernel. The same instructions like for getsequence (see above) apply.

API reference

Variable type mappings

typedef int            INT32;
typedef unsigned int   UINT32;
typedef double         DOUBLE;
typedef char           CHAR;
typedef unsigned char  UCHAR;
typedef	unsigned short UINT16;
typedef	signed short   INT16;
typedef void           VOID;
typedef	void  *        VOIDP; 

typedef unsigned long  uLong; 
typedef unsigned char  Byte;  
#define Bytef	Byte 
#define uLongf	uLong 

#define		TYPE_NONE_EMPTY   	-1
#define		TYPE_CHOOSE        	0
#define		TYPE_IMM           	1
#define		TYPE_IMC           	2 
#define		TYPE_BKG           	3
#define		TYPE_BKC           	4
#define		TYPE_SIGNIFICANCE   5 

#define		FCN_NONE           0
#define		FCN_AVERAGE        1
#define		FCN_ENVELOPE       2 
#define		FCN_SIGNIFICANCE   3 

struct VIDINFHDR
{
	UINT32	width;
	UINT32	height;
	UINT32	bpp;
	UINT32  bpp_effective;
	
	DOUBLE	scale;
	
	INT32	numframes;
	
	INT32	type; // TYPE_

	VOID  **framebits;
	DOUBLE *framescales;

};

In principle an image set is defined that all images have the same width, height, bpp, bpp_effective and scale values. The value scale is used as an easy way to access the scale value. The framescales values are only used for the sake of completeness and all of them should contain the same value as scale !

A one-dimensional array is the physical representation of the two-dimensional image data. The data is organized inside the array in the following scheme:
Imagine a image rectangle on screen. The image is stored in the array from top to bottom, line by line from left to right, pixel by pixel.

void initialize_vidinfhdr( struct VIDINFHDR *vinf );

This function is used to preinitialize a VIDINFHDR structure that was just defined. One must call it in order to use the structure afterwards.
Note: When you allocate your own frame buffer memory and afterwards construct a VIDINFHDR out of it, you have to use malloc() to allocate the memory. The new keyword must not be used.

void free_vidinfhdr( struct VIDINFHDR *vinf );

This function is used to deinitialize a VIDINFHDR structure that was used throughout the program. One must call it in order to free memory and clean up things after using the structure !
Note: The memory inside the structure which might be allocated is released by calling free() function. The delete keyword of C++ is not used.

INT32 load_file( INT32 type, const CHAR *filename, struct VIDINFHDR *vinf );

This function loads in any file type (with file name filename) created by the video system into the VIDINFHDR structure vinf. The type of the file is specified by passing the appropriate type of the file, either a proper TYPE_XXX if one knows the file format or TYPE_CHOOSE and the function will detect the filetype based on the extension of the passed filename parameter. The structure vinf needs to be initialized first and it needs to be empty.

• 0 successful operation
• -1 file could not be checked for type (by extension)
• -2 file type unknown
• -3 could not open file
• -4 error in file ! (ie. file corrupted)
• -5 file read error !
• -6 already initialized, please free VIDINFHDR (and memory) first
• -7 out of memory
• -8 decompression error

INT32 load_imm_file( const CHAR *filename, struct VIDINFHDR *vinf );

This function loads in the IMM file filename into the VIDINFHDR structure vinf. The structure needs to be initialized first and it needs to be empty.

• 0 successful operation
• -3 could not open file
• -4 error in file !
• -5 file read error !
• -6 already initialized, please free VIDINFHDR (and memory) first
• -7 out of memory

INT32 load_bkg_file( const CHAR *filename, struct VIDINFHDR *vinf );

This function loads in the BKG file filename into the VIDINFHDR structure vinf. The structure needs to be initialized first and it needs to be empty.

• 0 successful operation
• -3 could not open file
• -5 file read error !
• -6 already initialized, please free VIDINFHDR (and memory) first
• -7 out of memory

INT32 load_imc_bkc_file( const CHAR *filename, struct VIDINFHDR *vinf, INT32 type);

This function loads in either an IMC or a BKC file filename into the VIDINFHDR structure vinf. The type of the file is specified by passing the appropriate type of the file, either TYPE_IMC or TYPE_BKC respectively. The structure needs to be initialized first and it needs to be empty.

• 0 successful operation
• -3 could not open file
• -4 error in file ! (ie. file corrupted)
• -5 file read error !
• -6 already initialized, please free VIDINFHDR (and memory) first
• -7 out of memory
• -8 decompression error

INT32 load_imc_file( const CHAR *filename, struct VIDINFHDR *vinf );

This function loads in the IMC file filename into the VIDINFHDR structure vinf. The structure needs to be initialized first and it needs to be empty.

• 0 successful operation
• -3 could not open file
• -4 error in file ! (ie. file corrupted)
• -5 file read error !
• -6 already initialized, please free VIDINFHDR (and memory) first
• -7 out of memory
• -8 decompression error

INT32 load_bkc_file( const CHAR *filename, struct VIDINFHDR *vinf );

This function loads in the BKC file filename into the VIDINFHDR structure vinf. The structure needs to be initialized first and it needs to be empty.

• 0 successful operation
• -3 could not open file
• -4 error in file ! (ie. file corrupted)
• -5 file read error !
• -6 already initialized, please free VIDINFHDR (and memory) first
• -7 out of memory
• -8 decompression error

INT32 save_file( INT32 type, const CHAR *filename, const struct VIDINFHDR *vinf );

This function saves an image file (given by filename) out of the structure vinf (VIDINFHDR). One has to choose the type by itself (specify the right type, e.g. TYPE_IMM) or pass TYPE_CHOOSE. If TYPE_CHOOSE is passed, the contents of vinf (type field in structure) are checked and stored to a proper image file. In this case, the file extension is appended automatically inside the function after the file type was chosen. The image data inside the structure vinf will be checked for validity, e.g. a BKG file can only contain a single frame.

• type - gives file type, valid types are
  • TYPE_IMM - specifies an IMM file
  • TYPE_IMC - specifies an IMC file
  • TYPE_BKG - specifies an BKG file
  • TYPE_BKC - specifies an BKC file
  • TYPE_CHOOSE - choose type to save by content (type field) of vinf structure; add extension to filename automatically based on type!
• filename - specifies the location of the file, one should omit the extension if TYPE_CHOOSE is passed
• vinf - VIDINFHDR structure containing the image data to save

• 0 successful
• -1 multiple video frames can not be saved as bkg file
• -2 file type unknown
• -3 error in vinf structure (either bad type or bad numframes (i.e. 0) )
• -4 could not open file
• -5 file write error !
• -6 out of memory
• -7 compression error

INT32 save_imm_file( const CHAR *filename, const struct VIDINFHDR *vinf );

This function writes the image data inside a passed vinf (VIDINFHDR) into a newly created IMM file with name 'filename'.

• filename - specifies the location of the file and the filename
• vinf - VIDINFHDR structure containing the image data to save

• 0 successful
• -2 file type unknown
• -3 error in vinf structure (either bad type or bad numframes (i.e. 0) )
• -4 could not open file
• -5 file write error !

INT32 save_bkg_file( const CHAR *filename, const struct VIDINFHDR *vinf );

This function writes the image data inside a passed vinf (VIDINFHDR) into a newly created BKG file with name 'filename'.

• filename - specifies the location of the file and the filename
• vinf - VIDINFHDR structure containing the image data to save

• 0 successful
• -1 multiple video frames can not be saved as bkg file
• -2 file type unknown
• -3 error in vinf structure (either bad type or bad numframes (i.e. 0) )
• -4 could not open file
• -5 file write error !

INT32 save_bkc_file( const CHAR *filename, const struct VIDINFHDR *vinf );

This function writes the image data inside a passed vinf (VIDINFHDR) into a newly created BKC file with name 'filename'.

• filename - specifies the location of the file and the filename
• vinf - VIDINFHDR structure containing the image data to save

• 0 successful
• -2 file type unknown
• -3 error in vinf structure (either bad type or bad numframes (i.e. 0) )
• -4 could not open file
• -5 file write error !
• -6 out of memory
• -7 compression error

INT32 save_imc_file( const CHAR *filename, const struct VIDINFHDR *vinf );

This function writes the image data inside a passed vinf (VIDINFHDR) into a newly created IMC file with name 'filename'.

• filename - specifies the location of the file and the filename
• vinf - VIDINFHDR structure containing the image data to save

• 0 successful
• -2 file type unknown
• -3 error in vinf structure (either bad type or bad numframes (i.e. 0) )
• -4 could not open file
• -5 file write error !
• -6 out of memory
• -7 compression error

INT32 grabframes( const CHAR *servername, const UINT16 net_port, UINT32 num_frames, struct VIDINFHDR *vinf);

In order to acquire frames from any GrabServer2 program running, one should use this function. It is blocking and returns if either one of the error conditions is met or all frames have been acquired successfully.

• general return parameters
  • 0 all fine, all frames have been acquired
  • -1 unable to create socket
  • -2 unable to get hostname / malformed ip address
  • -23 vinf structure was not passed empty, please clean up the structure first
• errors that might be emitted by connect() call
  • -3 EADDRNOTAVAIL
  • -4 EAFNOSUPPORT
  • -5 EALREADY
  • -6 EBADF
  • -7 ECONNREFUSED
  • -8 EINPROGRESS
  • -9 EINTR
  • -10 EISCONN
  • -11 ENETUNREACH
  • -12 ENOTSOCK
  • -13 EPROTOTYPE
  • -14 ETIMEDOUT
  • -15 EADDRINUSE
  • -16 ECONNRESET
  • -17 EHOSTUNREACH
  • -18 EINVAL
  • -19 ENAMETOOLONG
  • -20 ENETDOWN
  • -21 unknown error at connect statement
• Errors that may happen after a successful connection attempt
  • -22 unable to switch to nonblocking mode
  • -24 could not receive video header stream
  • -25 could not receive frame data: timeout
  • -26 could not receive frame data: other errors
  • -27 unable to allocate memory location needed for storing frames
  • -28 frames are not in a sequence
  • -29 camera port change has happened during frame receive
  • -30 could not allocate memory for compressed frame bits
  • -31 could not translate vidhdr_stream into VIDHDR

Remarks: Any error after a successful connection attempt alters the vinf structure.

INT32 convert_format( struct VIDINFHDR *vinf, UINT32 newtype, UINT32 downscale_function);

This function is used to conveniently convert between different formats inside a VIDINFHDR. Most conversions are lossless, however, conversion of a VIDINFHDR with multiple images to FILE_BKG will be lossy because a downscale function is required and will be used.

• 0 okay, everything fine
• -1 inappropriate 'type' inside vinf structre
• -2 inappropriate 'type' parameter passed
• -3 downscale function selected which does not fit to passed structure and resulting type
• -4 convert to bkg, downscale function was NOT specified
• -5 unsupported bits per pixel settings inside vinf (using downscaling)
• -6 memory allocation error (out of memory, using downscale to bkg)

Remarks:
In any error case the vinf is not touched and the old data remains inside (without corruption) !
FCN_AVERAGE: averaging each frame pixel over the number of frames (average noise rules)
FCN_ENVELOPE: the maximum value of each pixel for all frames is obtained and stored as bitmap (max noise rules)

INT32 calculate_significance_map( const struct VIDINFHDR *sig_and_back, 
                                  const struct VIDINFHDR *back, 
                                  struct VIDINFHDR *out, 
                                  double sigma);

This function can be used to calculate a so-called significance map of two sets of matching frames (signal and background images). In case of no error, the resulting VIDINFHDR out will contain a single image with the same physical bpp as sig_and_back and back. A value of 0 means that the signal value at that position should be masked out (not significant). A value of 1 means that the signal value at that position should not be masked out (significant data). Using sigma (default value is 1.0) one is able to increase the significance of RMS of noise. Sigma is used as a factor to RMS. The bigger sigma gets, the more the rms of noise will increase and thus, more parts of the signal might be filtered out.

• 0 okay, everything fine
• -1 VIDINFHDR out was not passed uninitialized
• -2 sig_and_back must at least contain 2 frames
• -3 back must at least contain 2 frames
• -4 sig_and_back, back must contain the same number of images!
• -5 bitdepth, effective bitdepth, width or heigt of back and sig_and_back were not set to equal values (important)
• -6 memory allocation error, no data was changed (in out)
• -7 data error (unsupported bpp (physical value) for algorithm)

Remarks:
In any error case the vinfs are not touched and the old data remains inside (applies for any passed variable) !
The out vidinf structure must be passed uninitialized, otherwise you will get an error. Best is to use initialize_vidinfhdr before passing it to this function.

INT32 xrayfilter( struct VIDINFHDR *vinout );

Using this function one is able to apply x-ray filtering like the Video Client 2 does to a single frame or sets of frames. If this function is successful, it is lossy. The old data available inside vinout will no longer be available. It was overwritten.

• 0 okay, everything fine, data was changed
• -1 xray filtering failed (bad data passed in vinout), data inside vinout was NOT touched