Indian Innovators (12 page)

Ganesh (left) and Pragyanandesh (right)

VORWIS

Imagine being able to wear your personal computer on your eyes and carry it everywhere with you, interact with it via gestures (and not a keypad or mouse), with everything displayed in 3D, merging the real and virtual worlds.

This and much more, is what the duo of Ganesh and Pragyanandesh aim to achieve.

Ganesh comes from Vizag, Andhra Pradesh while Pragyanandesh (referred to as Pragyan) hails from Varanasi, Uttar Pradesh. Their journey to IIT has been as interesting as their travails after getting into the Institute.

Ganesh was an average student until Class IX, but he managed to get a good score in the Class X Board exams. His performance, along with a little luck, landed him a place in the topmost batch of the most prestigious IIT-JEE coaching institute in Vizag. There, he came into contact with some of the brightest people he had ever met and their company had a magical impact on him. By the end of the two-year coaching period, he was a “topper among the toppers” and was even sent by his coaching institute to Hyderabad for a 24-hour intensive crash course, just before the IIT entrance exam. Armed with immaculate preparation, he aced the IIT-JEE in 2009 with an All India rank of 121.

If Pragyan is to be believed, his academic performance in school was worse. He almost failed Class X. In Class XI, however, he developed some interest in academics for the first time, as he found the curriculum much more practical than in any of the previous classes. At that time, he was enrolled in a school in Bihar because his father, a Principal at Navodaya Vidyalaya, was posted there.

Pragyan became very inquisitive and started reading beyond his textbooks. His scores improved as well, which further fuelled his interest. Unfortunately, just as he was finally beginning to enjoy his studies, he had an accident on his way to school and fractured his leg. He was bedridden for the next six months and had to drop the year. This was a very frustrating and depressing period for him, but with the kind of determination he possessed, he was able to bounce back and did well in the class XII Board exams.

At that time, he knew little about IIT. Many of his friends were dropping a year to go to Kota and prepare for the IIT-JEE, so he too followed them to Bansal Classes. The preparation helped him develop a problem-solving attitude and made him realize that with hard work, he could achieve any goal he set his eyes on. The grueling work was not wasted, as he ended up with an impressive rank of 550, becoming one of the last few to secure admission into IIT Kanpur’s Electrical Engineering Department that year.

Ganesh and Pragyan met at IIT and struggled through the first year together, scoring 6 point GPAs. The academic environment at IIT was very different from what they had expected and they couldn’t adjust well. They found the course to be too theoretical, with parts of the curriculum being too outdated to have any practical value in today’s world. The “dull and boring lectures” failed to hold their attention.

Thus, they started exploring the plethora of extracurricular opportunities available at IIT and ventured into the Electronics Club. The opportunity to do more hands-on work at this Club caught their fancy.

With guidance from the club members, Ganesh made a GPS navigation system from scratch, in the summer break right after the first year. Meanwhile, Pragyan worked on several different things.

Pragyan recounts how they collaborated on their first project together. “In our second year, we worked together to make an improved version of the digital diary that could sync with any other device and could be fabricated in just
600. We presented the digital diary at
Aavishkaar,
the annual technology festival of Motilal Nehru National Institute of Technology, Allahabad. It won us the third prize in the competition. Though it was good for intellectual satisfaction, it didn’t make sense to pursue it beyond this point, because digital diaries were out of fashion.”

The same year, they developed a GSM-based electric billing system, which could be set up to monitor the power consumption of any device or house from anywhere, in real time.

Ganesh explains how the system worked. “It consisted of two parts. The first was the device that measured the electricity units consumed (like the conventional meter) and transmitted this information to a server via a GSM chipset. The second part consisted of the server and the website. Once the power consumption data is received by the server, it calculates the electricity bill per the billing rate built into it. This information can be viewed at the website in real time. The website can then be used to directly pay the bills. This is especially useful in places where there is a different billing rate for peak and the non-peak hours. So, people can monitor the billing and wisely choose how much to consume during the peak hours (when the rate is high). This is one of the facilities that smart grids provide in some western countries.”

They presented this idea at
Kshitij,
IIT Kharagpur’s technology festival and received a lot of appreciation. In fact, a professor from IIT Kharagpur approached them with an offer of
150,000 to commercialize it, but they declined the offer.

“We had seen a senior student do something similar on a commercial scale and fail. We were not too sure if there was a ready market for it as yet,” Pragyan explains. “We learned a lot through that project and felt that we would rather use that learning somewhere else, where we can make a marketable product, rather than taking this further and end up like that senior.

It was around this time that we came across Pranav Mistry’s work through his TED videos. His technology, popularly known as the SixthSense, is a wearable gestural interface that augments the physical world around us with digital information and lets us use natural hand gestures to interact with that information. It reminded me of the work I had done in my mathematics laboratory project in Class X, where I converted a mathematics textbook from 2D to 3D and found that learning through that textbook became much more enjoyable.

We were deeply inspired by Pranav’s work, but we found a few things that could be improved upon. For example, to use Pranav’s technology, one needs to wear special color- coded devices on the fingers. Moreover, the camera projects only 2D images. We thought about interaction with 3D objects in the virtual world, like we do in the real world.”

Thus, their project, VORWIS stands for Virtual Object in Real World, Interaction and Sharing. That’s exactly what the project aims to achieve.

To understand this some more, suppose you are playing a car-racing game on your computer. You see a car on your 2D screen and direct it via the arrow keys and accelerate its speed via the spacebar. Now, imagine a system that can be worn like eyeglasses. When you play the game on this system, it lets you experience a virtual 3D car as if you are sitting in it. You can control the car like you do a real one – by gesturing as if you are holding the steering wheel of this virtual 3D car and pressing the accelerator. Moreover, you can see the competing virtual cars around you as you vroom past them.

“With this idea in mind, we started working on the technology for creating a projection of 3D objects in air,” says Ganesh. After a month of work, we realized that we would not be able to do it without Virtual Retina Display, a technology that is still under active research across the world. Meanwhile, we still had to figure out how people would interact with that virtual 3D object.”

Virtual Retina Display (VRD) is a technology developed by Human Interface Technology Lab, Washington DC, USA. The technology aims to bring vision to those who have damaged eye lenses but a working retina. When the colored light obtained from the reflection of white light from a particular object is focused by the eye lenses on the retina, certain cone and rod cells in the retina are activated, which help us perceive that object. Using VRD, the same rods and cones are activated via low intensity colored laser beams which can be directly focused on the retina, enabling vision.

Pragyan says that they decided to go ahead with their basic idea. “In January 2011, we decided to apply to the India Intel Embedded Challenge. We did not have much hope, but did it just for the sake of it, because there was no harm in trying. Thus, our first abstract on the ‘Technology to interact with virtual 3D objects with gestures’ was submitted.

In February, we submitted the final abstract and it was selected. The funny part was that we still had no idea how we would realize what we had hypothesized.

Interpretation of gestures required capturing the gestures via a camera and then developing software codes to identify each gesture and perform the action intended by it. We had no idea which camera we should use to capture the depth of 3D objects or actions. Around the same time, Microsoft came out with the Kinect for Xbox 360. The Kinect sensor captures 3D motion, as it is capable of recording the depth data that normal cameras cannot.

We heard that the Computer Science Department at IIT Kanpur had bought a Kinect and decided to approach a professor at the Department to let us use it. We were not too sure if permission would be granted. However, the professor immediately arranged for all the required permissions when he heard what we intended to do with it.

We were already in May 2011, and were running short on time,” Ganesh adds. “But that’s what you learn at IIT – working toward tight deadlines. Once we had the Kinect sensor, we started working on the library of software to enable recognition and interpretation of gestures.

Because we wanted to implement direct recognition of gestures without the need for any additional devices or color coding, the algorithm was very complex. Further, we also did not intend to project the graphical interface on a 2D surface, like in Pranav Mistry’s work. This added to the complexity.

Anubhav Singla, one of our seniors and a coordinator with the Electronics Club, helped us a lot. He had been part of the Intel Embedded Challenge earlier and gave us useful insights. Finally, in August 2011, we made the final presentation on our work in front of a jury. They were deeply impressed with what we had achieved and adjudged us the winners in the Educational Technology category. Though the competition ended, we continued with our work, using
50,000 that we won.”

Pragyan explains how the technology works. “With our system, you would be able to see the 3D image on special eyeglasses. You would also be able to view the real-world objects around you at the same time through those glasses. Moreover, you can take a real object into the virtual world, if you come within a certain distance of that object and perform a certain action. Say, you want to solve an unsolved Rubik’s cube lying in your room. You can go within a certain distance of the Rubik’s cube, take a 3D picture of that cube, and then solve it as a virtual 3D object by using your hand gestures, like you would for the real cube. So, you have solved that Rubik’s cube without even touching it. It still lies unsolved, at the same place; yet, you have done whatever you intended to do with it.