• Apps in the Living Room: An increasing number of televisions are connected to the internet.  Many of these have pre-installed applications to access YouTube, Netflix, Pandora, and other popular services.  Internet connected television platforms from Boxee, Google, Yahoo,  LG, and others make it possible for viewers to install additional applications by browsing app directories or bookmarking TV optimized websites.   It is an open question of which “lean back” apps will prove popular with consumers.  In the next twelve months we will have a better idea of the types of apps that are right for living room versus those that are better suited for single users on a PC or mobile device.

• Further Definition of the New Music Industry: Overall, the music business will continue its ongoing process of consolidation.  The notion of owning music in the form of a CD or a digital download will continue to erode, particularly for Generation Y, being replaced by cloud driven subscription models.  Internet data driven charts such as those powered by BigChampagne and the Echonest will rise in influence as sales and airplay based charts lose relevance.  Many of my thoughts are echoed in this Hypebot interview with Music 3.0 author Bobby Owsinski.

• Interactive Retail: In 2010, shoppers used smartphones to compare prices, earn coupons, and even make purchases.  This trend was encouraged by services like Foursquare, SCVNGR, and the use of QR Codes, Microsoft Tag, and other visual barcode systems.  As smartphone ownership continues to increase in 2011, marketers and advertisers will further use these technologies to bring new levels of interactivity to the retail experience.  We can expect  more companies to offer custom applications similar to  this one from the Carl’s Jr. and Hardee’s restaurant chain.

• More Tablets: By all accounts, the iPad was an enormous success.  This year more Android based tablets are expected to be released along with ones  from Blackberry and Palm.  With a new iPad being released this year, competing tablets must be aggressively priced and provide compelling app offerings to gain significant market share.

• 3D Backlash: 3D was heavily pushed by the film and consumer electronics industries this past year.  There was an over abundance of 3D film releases in 2010, and a number exhibited poorly implemented stereoscopic effects.  Consequently, movie goers will be more reluctant to pay extra to see a film in 3D.   Though 3D capable televisions were touted to consumers, there is a lack of 3D content for these devices.  Unless the content availability issue is resolved, 3D television will be of interest to only the very early adopters.

• Kinect Hacks: For years, researchers have utilized computer vision to build gesture based interfaces.  An excellent example is seen in the M.I.T. Media Lab’s Smart Rooms projects in the 1990′s.  Depth sensing cameras could cost in the $30,000 range, a price which prevented many developers from exploring this area of interface design.  The Microsoft Kinect is an amazing piece of hardware in that it provides a highly functional depth sensing camera system for only $150.00.  Realizing its potential, hobbyists and researchers have quickly built up a thriving Kinect hacking community in similar fashion to what occurred with the release of the Wii.  With open source efforts now springing up to support the Kinect, this community will continue to thrive and innovate in 2011.  Their work will hopefully inspire and guide the design of innovative gesture driven games and applications for the mass market.

Using mobile apps like those from Ustream and Qik, a few are even live streaming the show. Taking note of this behavior, a compelling opportunity exists to use these devices to create opportunities for fans to interact with the on stage artist and become participants in the performance.

One of the most ambitious efforts in this area is being led by techno DJ and producer Richie Hawtin. An established music technology innovator, Hawtin gained attention in recent years for tweeting his DJ sets. As part of his current tour performing as his alter ego, Plastikman, he and his collaborators have released the “SYNK” application for the iPhone and the iPod Touch.

During performances audience members can use the app to see real time data from percussion gear, live video from an on stage perspective, and the application will synchronize phone visuals with activity on the stage LED wall.  More details on the app can be found here.

Here’s video of the “Synkotik” mode of the app.

In this LA Weekly Coachella interview, Richie Hawtin describes the motivation for integrating SYNK into his performance.

“I think a great concert experience is when unexpected things happen and when people come together to experience something that goes beyond music. The idea was to throw something into the mix that would heighten that experience. With the app we’re able to send people messages, we’re able to actually synchronize things that are happening on stage with the iPhone apps, we’re even actually allowing people to actually have access to my set-up, you know, triggering sounds. So everyone talks about technology democratizing everything. With our app we will actually blur the lines between the performer and the audience, actually forcing questions of, who is the performer, who is being in control, It is an experiment.”

In addition to the work of Hawtin and his collaborators, it should be noted that there are other efforts in this area, such as Eyebeam’s Open Video Sync project.  It aims to create an open source codebase which will allow multiple iPhones to synchronously playback video using Bluetooth or Wifi transmitted timecode.

Using smartphones apps for interactive concerts is a concept that is still in its infancy.  It is clearly a powerful avenue for tech savvy artists to push the boundaries of live performance.  Furthermore, by creating a way for audiences to participate in the performance, fans are given an additional incentive to buy tickets, an opportunity which cannot be overlooked in today’s music industry where artists are increasingly dependent upon live show revenues.

Researchers at the University of California San Diego Computer Audition Lab have developed a system that could address this problem by allowing people to find music using descriptive words rather than artist names and song titles.  For instance, a person could enter the words “high energy guitars” or “romantic vocals” and then receive a list of tracks that match that description.

The USCD system is capable of ingesting songs and automatically tagging them with annotation data without human intervention. To provide accurate results, the system must first be taught to hear music and describe it using natural language.  The training process uses digital signal processing and machine learning algorithms to expose the system to a broad array of music along with the words people use to describe it.  For example, to be able to accurately identify music that is referred to as “driving rock”, the system must analyze a large number of driving rock songs and then identify signal patterns that make that particular style of song unique.

The researchers have been gathering training data through crowdsourcing using an innovative Facebook game called “Herd-It“.  In this game, users are played a song snippet and asked to associate descriptive words and phrases with it.  Users earn points based on how well their answers match those of previous players.  Here’s a video describing the game.

The research group’s latest work in improving automatic music analysis was recently presented at the 2009 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) in the paper “Dynamic Texture Models of Music,” by Luke Barrington, Antoni Chan, and Gert Lanckriet.

With the continuing decline of the radio DJ as taste maker, web based music search and discovery tools will become increasingly important. With further development, machine learning driven music search systems such as this one could provide an intuitive and compelling method for listeners to find music they will enjoy.

Here are video captures of the performances.

This promotion is an excellent example of using innovative technology and engaging content to capture audience attention. Simultaneously, it provides a unique avenue for artists to promote their music and live shows.

Augmented reality (AR) is a technology that has been around for quite a while, primarily in the academic and research domains. Recently, its mainstream presence has increased due to the development of the Flash version of the Augmented Reality Toolkit (FLARToolkit). The ARToolkit was developed by Dr. Hirokazu Kato with Dr. Mark Billinghurst at the University of Washington’s Human Interface Technology Lab (HITLAB) over ten years ago.

In 2000-2001, I led a team which modified the original C based ARToolkit to work on 3D accelerated desktop Windows PC’s. We used the toolkit to develop interactive augmented reality projections for the band Duran Duran’s live concert tour. Here’s a video of ARToolkit effects that we used in the live shows.

The project was documented in this presentation at the 2002 Augmented Reality Toolkit Workshop:

Jarrell Pair, Jeff Wilson, Jeff Chastine, Maribeth Gandy. “The Duran Duran Project: The Augmented Reality Toolkit in Live Performance”. The First IEEE International Augmented Reality Toolkit Workshop, 2002. Download PDF

Now that it’s possible, what will be the breakthrough applications for mobile live broadcasting?   I think the answer may lie in looking at the trend of celebrities using Twitter.  Twitter is popular with stars because it is simple and easy to maintain.  It can be almost spontaneously updated unlike a traditional blog or personal website.  Celebrities can easily enhance this fan communication channel using Ustream.tv, Qik, Kyte, or Flixwagon.  In particular, mobile broadcasting could be appealing to touring musicians who rarely have an opportunity to sit at a computer and send a well composed personal blog post. Rapper Soulja Boy has been an early adopter in this area by using Kyte’s mobile platform to keep his fans in the loop.  In a similar fashion, Lil Wayne is set to begin using Ustream.tv’s mobile application.  Mobile broadcasting is clearly a concept to keep an eye on over the next year.

Here’s Ustream.tv’s mobile demo video.

Microsoft Tag has recently become publicly available at http://tag.microsoft.com/.  Visit http://gettag.mobi/ to obtain a reader for most camera equipped smart phones.  Potential applications for media distribution, promotion, and advertising are vast.  For example, a band could enhance their show flyers with tags linking to online ticket offers and guest lists.  A couple of years ago I experimented with Semacode, a similar technology, to develop demos showing how tags could be used to distribute music videos and film trailers.

Demo 1: Music Video Distribution With Visual Tags

Demo 2: Film Trailer Distribution With Visual Tags