As the semester draws to a close, CS349 has been a  very intense but inspiring class. One of the major accomplishments that I feel I have done (along with my group, of course) is complete a functioning prototype, that reflects a lot of work and learning as well as a lot of thought and innovation. I recently looked over the project (and fixed filtering, exciting!) and was astounded and how much of the project we were able to implement in less than a month (we started implementation of the functional prototype after the second milestone officially). As well, the project has changed drastically in look and feel for the better. The various steps we have taken, the different milestones, have been a great introduction to the methods and tools that developers use to move towards a working product. The initial brainstorming, the use scenario, the paper prototypes, etc. have all been important to the development of our project.

Our project implementation only merely scrapes the surface (pun not on purpose..) of the possibilities with tangible user interfaces, which is what makes TUIs such an interesting field of study. My hope is that our project will be beneficial to future developers, and that it may be further developed into a working product.

Until next time TUIs,

Donna Yee

http://vimeo.com/8126974

Today after the presentations, I managed to allow users to filter notes using the panel. However, at the moment, there is no way to remove the filter. Ideally, users would be able to toggle that off using the panel. Unfortunately removing items from the list is more complicated than adding, so I plan on working on that a little bit. With the ability to filter notes properly however shows us that it is also possible to remove that option.

I would like to see future work done/even do future work myself on this project. I think this type of project could be expanded to other forms of performance groups such as a cappella or dance groups very easily, and it would be great to see that.

Main Page of Dynamic Rehearsal

The current interface of Dynamic Rehearsal is much more aesthetically appealing and more tangible than before. While we still have GUI-like elements in our design, the elements that distinguish our interface from being purely GUI-like is the collaborative aspect as well as the scatterview objects inherent characteristics. Ideally, all elements on the Surface in our interface can be moved, rotated, and resized (not currently fully implemented). This allows for a viewing multiple rehearsals, more collaboration, and fun times.

Here are specific elements of our interface:

Individual Note	Multiple Rehearsal Items
Note Selector, Timeline and Video	Recent Notes

As you can see, there is a great deal of information on the interface at any given the time. The flexibility to move would allow users to focus on what they want to focus on (they could throw the other elements elsewhere and/or drag the element they wish to focus on to their center of their view.) rather than become overwhelmed by all the information, while still giving users the ability to retrieve all the information.

I was able to fix the filtering method so with more time, we would be able to generate the selector panel, legend and play information on the fly and have the option of filtering notes. Currently the selector is not interactive.

We also have a canned interaction with a note so that when clicked, it will open up an individual note on the side. This demonstrates the possibility of developing the interaction fully with more time.

Tasha added a seek slider that interacts with the movie to display a different point of the movie. The time at which the movie is moved to is also displayed though this does not dynamically change as the movie is playing. Also there is not an interaction with the three different components of the interface: timeline, movie, and notes, though these three separate components are developed pretty fully.

I also managed to fix filtering so that it is possible to filter notes. However there is not currently a way for users to to interact with this method.

We have included actual rehearsal video and notes to give a more realistic feel to the application.

• generate each note as a button within the stack panel
• have the side icons matching each actor’s color (and have multiple show)
• have side icons match note type
• have multiple icons show
• begin a possible filtering method
• be able to scroll the notes panel
• order notes by time

I have managed to implement an interactive scrollPanel with each note displayed as button (for interactive purposes later) with icons matching multiple actor colors (however not shape). The notes are ordered by time, and I have an inspiration for doing the filtering. I managed to, in a sense, filter the notes by type, but not from the user end of the interface. Problems I encountered included:

• Notes not parsing correctly (format error involving, most likely, hidden characters)
• Having multiple icons display. Had to come up with an algorithm that could position multiple icons depending on how many there are. Simplistic math equations, but certainly stumped me for a bit.
• In general, creating the scrollPanel. It was a steep learning curve in learning how to actually generate complete scrolllers and whatnot, which we had just hardcoded in through xaml.

Still to solve:

• Displaying different shapes. For whatever reason, displaying different shapes did not work well at first. Also, my code currently generates different images than what is on the timelines so these must reconciled.
• Filtering by people. The filtering by different types of notes does work currently, though users cannot access it directly. However, I still need to work on the code for filtering by actors. At the moment, the code is very inefficient so ideally, I would be able to optimize the runtime of code.
• Filtering by users. The next step in filtering would be to allow users to filter the notes. This would involve generating a new notes panel according to their selected buttons and replacing the current notes panel.
• Individual notes. We also decided during our redesign it would be helpful to able to select different notes and pull them out in an individual note for later use. Hopefully it will be a drag and drop movement, however a click interaction wouldn’t be a problem either.

Yes, lots to do and not a whole lot of time. Off to work more!

Old DR layout

New DR Layout (rough draft)

Our hope is that this new layout will be less cluttered and more aesthetically appealing for the user. Not only that, one of the draws for this type of layout is if it were placed on a scatterview, one could open multiple rehearsals and be able to compare that, allow other users from across the table to see it, etc. This layout definitely makes it more apparent that users can move it around.

One of the things we struggled with is the fact that the design is still very much like GUI. The main difference between this application and just a simple GUI application is the collaborative aspect of it. And we want to take advantage of the tangible component of the surface by adding the ability to move each individual rehearsal item (hopefully!). Still, there are many GUI-like components of the interface that later implementations can improve upon.

Current Dynamic Rehearsal Implementation

For the horizontal and feasibility prototype, we chose to focus on implementing the timeline section of our application as it requires generating images on the fly. We have been able to implement note parsing, generating the note markers, and time blocks for the time line as well as outline our original design layout* Major changes to come. In addition, we successfully added a video without controls.

MY DEVELOPMENT

I focused on parsing notes and creating the not markers on the timeline (with Tasha’s markers created). As well, I added in the very minimal movie player.

Notes:

To parse the notes correctly, we assumed formatted notes and that the pen automatically converts to a text file into this format. Our format relies on tabs to delimit different characteristics of notes (timestamp, content, etc.) and on lines to contain only one note. We create new notes from the information we extract and for each person for whom the note is written, we assign the note to the person, thereby generating markers on the timeline.

The next step in implementation of notes is generating the notes panel on the fly. With the timeline already being generated, there will be a similar structure to the code. Furthermore, we also want to add sorting ability to the notes panel such as choosing to look at only certain types of notes or certain people.

Conceptually, parsing the notes is not difficult once broken down. However, we had difficulties with the text files being wrongly formatted despite visually not seeming to be so. There still remains more challenges with the generating of the notes panel as well as sorting notes. In our new design, we hope to able to implement the ability to drag a note out into an individual object.

FUTURE WORK

I will continue working on notes, specifically generating the notes panel and adding controls to the movie such as a time marker for the timeline and pause/play controls (with Tasha as she has worked on the timeline). We are currently in the process of rethinking and redoing our layout design, allowing us to take advantage of the tangible aspects of the Surface. This will be detailed in the next post.

In a slight contrast of Weiser’s vision, the writers of “Tangible Bits: Towards Seamless Interfaces between People, Bits, and Atoms” describe a vision where digital information can be mapped to any and all physical objects. While their vision also includes the idea of ubiquitous computing, the authors approach it in a different way. Rather than have pads and boards all with computers, they look into having different physical objects that have computers integrated into them. For example, the marble answering machine, which we have previously discussed, allows users to interact with nonphysical information (voice messages) by representing them as marbles. Live wire is also an interesting idea that allows users to “see and hear bits” represented. Digital information, because of its inherently nonphysical attribute, is difficult to represent to users. With ubiquitous computing in this vision of tangible bits, it can be represented by everyday objects.

Both articles describe a vision of ubiquitous computing that fundamentally changes the idea of personal computers. Dynamic Rehearsal certainly embodies some characteristics of the vision of ubiquitous computing. By combining common physical objects such as the pen and camera with the surface, a computer combined with a table top, Dynamic Rehearsal demonstrates some of the benefits of such a combination. An example would be the organization of information and the more knowledgeable system that, in turn, informs users. Dynamic Rehearsal seems to align more with the vision of Wesier, where computers are embedded into pads and boards. I imagine a vision that aligns more to the Tangible Bits idea would be having the physical objects return the digital information rather than the surface. In some ways Dynamic Rehearsal is limited in that there is till the focus of the user on a screen. Though the screen is embedded into a table top, partly because of the cost, it is still seen more as a large screen computer.

The two visions of ubiquitous computing can easily coexist and are certainly ones that we are moving towards. It is exciting because it offers new dimensionality to previously static things such as a desktops and walls. The exchange of information is enhanced greatly and collaboration is much more easily done with ubiquitous computing.

Christina Aguilera, Keeps Getting Better