Interactive systems with artificial vision: C++ openFrameworks programming workshop

18:3021:30 h
Venue: Hangar
Emilia Coranty, 16 – Can Ricart
08018 Barcelona
Price: 200€

The desire to deposit in machines physical and cognitive functions similar to those of humans can be traced back beyond the Middle Ages and can still be seen today updated in artificial vision technologies, capable of perceiving, interpreting and digitizing subjects and objects of the physical world. This course investigates in a theoretical and experimental way the creative potential that artificial vision offers for the creation of interactive works and research in corporal performativity in virtual and augmented reality environments.

We will start from the study of the relationship between bodies, machines and performativity, encompassing references in phenomenological, materialistic and posthumanistic thought of the cognitive sciences and human-computer interaction, and the new philosophical conceptions of the natural and the artificial, the being, the presence, the sensory immersion and the limits of the digital body. We will program in C++ language using openFrameworks oriented to develop applications to acquire and process data from video cameras and sensors such as Kinect v1, v2 and Leap Motion, being able to detect facial gestures and body movements for the control of audiovisual events in real time.

Participants will design interactive experiences that allow the simultaneous participation of several users interacting with the system using their body and gestures in a transparent and natural way. They will build prototypes of interactive systems in mixed groups of novice and experienced programmers, designers, theorists and performance artists, culminating in an exhibition and assessment of the results.

Aimed at: Interested in performing arts, live audiovisual, installation and stage design, costume design, music, advertising, television. 3D, virtual and augmented reality developers. Researchers in education, psychology and cognitive sciences.

– Study of the philosophy of the digital body and the aesthetics of the interactive work of art.
– Aesthetic and technological analysis of complex interactive systems.
– Programming skills with specialization in machine vision and machine learning using the C++ language and OpenFrameworks.
– Design and build a prototype of a controlled interactive system with live gestures and body movements.

Requirements: The workshop is oriented to people with little or no experience in C++ / openFrameworks programming. It is only necessary to attend with a laptop computer. Participants are invited to bring project initiatives to be developed during the course.

Course contents:
– Studies of the evolution of the relationship between body, machines and performativity, covering historical references from the fifteenth century, the idea of the total work of art, phenomenology and cognition with the body, to the current paradigms of virtual reality, artificial perception and immersive systems.
– Analysis of the thoughts of Birringer, Virilio, Deleuze, Laurel, Haraway, Merleau-Ponty, Braidotti, Papert, and others.
– Archaeology of computer vision and machine learning. Fundamental devices, techniques and algorithms, and their main areas of use. Studies on interactivity. Interactive experience design methods based on technologically enhanced body and space.
– Technical analysis of an interactive installation: sizing, arrangement of lights, cameras, sensors and audiovisual systems. Overview of software and hardware for capturing images and controlling multiple synchronized projectors.
– Programming in C++ oriented to artificial vision. Image capture, pixel by pixel operations and the main algorithms of analysis and prediction. Advanced real-time image processing: color detection, motion, contours, faces, pattern recognition using the OpenCV library.
– Methods for acquiring and processing data from Kinect V1, Kinect V2 and Leap Motion sensors. Data interpretation, user detection, movements and contours, skeletons, depth maps. 3D scanning of objects and people using Kinect.
– Control sounds, 3D graphics, and OpenGL shaders in real time through gestures and body movement. Interaction with 2D/3D mesh networks, Voronoi structures, Delaunay triangulation, and advanced particle systems using gravity simulation, attraction forces, repulsion, waves, navier-stokes for 2d fluids.
– Audiovisual control through gestures and body movements: image sequences, reproduction and manipulation of films and audio samples. Application of VJ/DJ techniques, sampling and audio video synthesis synchronized to body movement.
– Import and export XML, JSON, OBJ, STL files. Networked installations: Network connections via OSC Open Sound Control or MIDI, Syphon.
– Introduction to artificial neural network concepts and Machine Learning techniques with interactive or real-time uses for the interpretation of patterns, facial gestures or whole body movements and other real-time data.

Duration: 24 hours

Dates: 2, 3, 4, 5, 9, 10, 11, 12 July 2018

Time: 18:30 – 21:30

Registration Fee: €200

In charge of: Laboratory of education in art, technology and philosophy

The registration form can be found at the link of Hangar .

Sistemes interactius amb Visió artificial: Taller de programació C++ openFrameworks