A Conversation with ParaGraph’s Stepan Pachikov

When people pick up an Apple Newton, for better or worse, much of their impression of the product is based on the handwriting recognition system that has been made so integral with the product’s identity. And that has put the handwriting system’s developer, Moscow-based ParaGraph International, in an interesting position.

However, speaking with the company’s 45 year old president and CEO Dr. Stepan Pachikov recently, it’s clear that ParaGraph has overcome a number of challenges—from starting a high-tech company in a communist country to licensing its technology to Apple Computer.

Pachikov has an impressive and diverse background. He recalled: “I have been in business for the last four years. In my previous life I was a computer scientist. I have a Ph.D. from Moscow State University in fuzzy set logic and fuzzy languages. My Ph.D. dissertation was about developing technologies and software to control robots on Mars or on other planets using fuzzy languages, and natural language commands. My other background is in economics.”

His business background, however, started far from mobile computing. He recounted: “When I was a student in Siberia, I organized maybe the first rock group in Russia, in Siberia. It was a real big group at that time. I was initially a drummer in the group, and afterwards I became a manager for the group.”

The Genesis of ParaGraph

ParaGraph started as a company about five years ago with the initial name of Micro-Contour. Pachikov remembered: “Our goal at that time was just to escape from the official Russian science structure, because of the Russian bureaucracy. We tried to form our own, small, Russian Bell Labs.” However, they lacked the support of a large organization around them and quickly decided to build products to become self-sufficient.

Pachikov reminded me of the context in which they had to work at the time: “Don’t forget, it was 1988, it was the Soviet Union, and it was the beginning of Perestroika. There was no capitalism, and only small signs of private economic cooperatives. But Gorbachev came in with the idea of joint ventures. However, joint ventures in Russia at the time could be organized only by foreign companies and Russian legal boards, not by private people.”

As a member of the Academy of Science, Pachikov was able to draw from a rich pool of talent, and explored options in setting up a joint venture. Pachikov explained: “We found a young and very interesting guy from Oklahoma named Scott Klososski who was working in Moscow and looking for something to do. He was 27 years old and we organized a joint venture with him named ParaGraph, and this is how the company started.”

Getting ParaGraph Noticed

Pachikov continued the story: “We had no money, and he [Klososski] could not find the money or [outside] interest to invest in the company. But having the name and having the status of a joint venture gave us a lot of leverage to ask four or five leading people, the most recognized in the software industry in Russia, to become members of our team.”

“Finally, we organized our company and immediately signed a small contract that gave us $40,000, which was a lot of money at that time. We brainstormed about what work to do.”

The time period was 1989 and ParaGraph had to find its focus. Pachikov explained: “We made a list of the 25 most interesting problems in artificial intelligence: cursive recognition, voice recognition, shapes, earthquake prediction, and some problems which we had worked on in our previous scientific lives.”

“We decided that maybe the easiest for us was to develop cursive recognition because it didn’t require any equipment. We had very impressive technology at that time for predicting lung cancer, but it required a lot of X-ray equipment.” Pachikov joked: “This is why we started to do something simple like cursive recognition.”

Having settled on a project, Pachikov needed to develop a justification. He noted: “I started to think that I needed a marketing plan, a marketing vision. Don’t forget that I was not a businessman.” Having read several books by the famous Swiss psychologist Jean Piaget, Pachikov thought he had found his angle.

“Don’t forget, it was 1988, it was the Soviet Union, and it was the beginning of Perestroika. There was no capitalism, and only small signs of private economic cooperatives. But Gorbachev came in with ideas of joint ventures”

“I said to my friends that we have to develop a handwriting technology to motivate children to write, because this is a very important way to keep their IQ high.” He also reasoned that by making it into a game, children would spend more time with the program and parents would feel justified in purchasing it.

He noted: “I know parents that will agree to pay any money just to have their children’s IQ a little higher. Plus we have to develop a guilt complex with the parents that if they don’t use our handwriting software, their children might be in a dangerous situation. That was my first marketing plan.”

Summing up, Pachikov noted: “this was our motivation to develop handwriting. No GO machines, no Newtons, no digitizers, nothing, because we had no idea about digitizers at this time. Don’t forget, it was Russia with no computers. The only computer we had in our company was my old Amstrad.”

He recalled: “This is why, because we had no idea about digitizers, that we decided we had to create a recognition technology for off-line, scanned images. And children would have to write and scan, or something like that.”

Moscow Children’s Computer Club

Pachikov proudly recalled how he met many foreign computer dignitaries visiting Russia in the late 80’s. “I was a founder of the Moscow Children’s Computer Club in 1986, the most famous and prestigious children’s club in Russia. Fortunately, I am a friend of Gary Kasparov, and in the children’s club I met a lot of luminaries: Seymour Papert, Phillipe Kahn, Esther Dyson, Mitch Kapor; everybody who was in Moscow.”

He continued: “This is why, when Jerry Pournelle from Byte came to our company, we asked him to write something for our recognition technology. I have a lot of samples of handwriting of famous people.”

Pachikov started showing demos of their recognition software to visiting people using an 8086 computer and scanned images. He explained: “We then decided to take it to CeBIT, the large computer show in Hannover. We came to CeBIT with no experience, no booth, nothing! We just had our suitcases, trying to organize a demo of our software. Because I knew a lot of people, I found people from Ashton-Tate and they remembered me.”

“They remembered our demos and they were very impressed. So they let us demonstrate our software in their booth, at Ashton-Tate in hall number 7. And it was the most prestigious hall there, including Microsoft, IBM, Compaq, everyone!”

Following CeBIT, Pachikov decided that it was time to take the demo to the United States. He recalled: “We came to Spring COMDEX in Atlanta, Georgia, on May 30th. It was the day when Gorbachev came to the US. And if you remember, it was Gorby-mania. We had our own booth in Comdex along with a Soviet booth, with a red flag, hammer and sickle.”

ParaGraph’s 12 Minutes of Fame

COMDEX and Gorby-mania proved to be a nice match. “We were invited to all the receptions, and the chairman of COMDEX invited us everywhere. CNN even spent twelve minutes reporting about our company. It is really funny: Russian missiles, submarines, and a handwriting company with me speaking. I speak terrible English now, but at that time, you have to see this interview, it was unbelievable because from the beginning of our company I didn’t speak English at all, I used interpreters. At COMDEX, it was the first time I spoke English by myself during the interview.”

Pachikov recalled an interesting moment at that COMDEX. “I remember that when I came to my booth they had just finished making the report and packed their equipment. When I arrived, I said just one or two phrases that the real reason for Perestroika wasn’t Gorbachev, but it was the microprocessor revolution. I said that Perestroika started in 1971, in Silicon Valley, when Intel invented the microprocessor.”

“They looked at me, unpacked all their equipment, and asked me to say it again. I said to them that all Russian power, Communist power, was based on control of the military and control of information. And I said that microprocessors, more than telecommunications, keep the control of information and Communism cannot exist without that.”

Online Recognition

At Spring COMDEX, people were impressed and surprised, wanting to know why the company had selected to recognize scanned images instead of input directly from a digitizer. Pachikov thought: “We didn’t understand why it would be difficult; if we could recognize from paper, why is it a problem to recognize from a digitizer?”

Of course, it’s a different problem altogether, but Pachikov claims online recognition has some blessings. “It is different, but it is simpler because you have more information. The only difference between writing on a digitizer and writing on paper, with scanned images, is that [with paper] you lose a lot of information about pressure, and other things.”

To perform online recognition, however, Pachikov needed a digitizer. “In New York I have another friend called Herb Hauser. His name must be mentioned because I’m sure that a portion of our recognition success and a portion of our Newton glory is connected with his name, because he gave us a digitizer. Using his digitizer, we developed an online version of our recognizer, which we brought to Fall COMDEX in 1990 in Las Vegas.”

At Fall COMDEX, ParaGraph’s technology worked, but not without restrictions. Pachikov remembered: “Mitch Kapor came to our booth, and we gave him a demo of our online recognizer using a digitizer. First there was a one word recognizer, only one word, no sentences. Words had to be written in the middle of the digitizer, fixed size, no rotation, with a dictionary of about 4700 words.”

Recognizing the Beatles

A former musician, Pachikov incorporated his previous avocation into the demo. “For the dictionary, we had no computer version for English words, but one of my friends had, on floppy, old Beatles songs. So we just took the Beatles songs, which I now understand has a limited vocabulary, and we turned it into our dictionary.”

Pachikov continued: “Mitch Kapor came, wrote a couple of words, and fortunately our recognizer recognized his handwriting. Mitch then started to call almost everybody in the United States. He phoned from our booth, called Jerry Kaplan, and said: ‘Jerry, it really works, you have to see it.'”

Right from the beginning, ParaGraph concentrated on connected cursive. ParaGraph’s next step was to work on showing it around. Pachikov recalled: “I had met Al Eisenstein in Moscow, so I phoned Al from our booth, and told him: ‘Look, we have this cursive recognition technology, and Mitch Kapor likes it very much.'”

Looking back, Pachikov related that: “I had absolutely no ideas, zero, about pen computing; GO machines, Newton, nothing. But it was a gut feeling. It’s difficult to explain why I decided to call Apple. No articles, no information. But from a lot of people I knew, I decided to call Apple.”

He continued: “The reaction from Al [Eisenstein] was unbelievable. And he’s a vice president at Apple, so it’s not very easy to get him on the phone. I got a call a few minutes later and he said that I had to go to Silicon Valley to see Larry Tesler. So we came to Silicon Valley, for the first time in my life.”

The Russian From Oklahoma

Pachikov had made an earlier trip to visit his American partner Klososski. While there, he somehow managed to convince local authorities to issue him an Oklahoma driver’s license. He recalled: “I got my Oklahoma license, which I used for the next two or three years, and everybody was surprised everywhere when I showed my documents, it was from Oklahoma.”

Continuing the story, Pachikov noted: “So, we came to Silicon Valley and started to visit companies. We visited NeXT and I had a meeting with a vice-president, Bud Tribble. We explained to him about our recognition, but we were told that Steve Jobs did not believe in pen computing and that he was very skeptical about it.”

“I tried to convince him by saying that NeXT is a great, excellent machine for desktop publishing, but handwriting recognition may be the key element for desktop publishing. It is very important not for writing text, pen computing is not for writing, pen computing is for editing. The pen is not only for input, the pen is an instrument with which to think. It’s something which must be in your hand when you have to manipulate for editing, and deleting.”

Disclosing at Apple

After little success at NeXT, Pachikov moved on to the leading mobile computing companies of the day: “Finally, we met Jerry Kaplan, Robert Carr, and other people from GO and gave them a demo. We met people from Intel, and a lot of companies here [in Silicon Valley], almost everybody.”

“But finally we came to Apple; to see Larry Tesler, Steve Capps, and a few other people. We gave them a demo of our recognizer, using our digitizing tablet and a PC. First of all, as I remember, our system recognized nothing! Everything Larry Tesler tried to write gave the wrong answer.”

“But they asked us how it works, and we started to explain it to them. We gave them a demo of all the key elements, and we explained to Larry that we try to recognize without context using the dictionary that we wrote ourselves. And it looks like they were impressed, maybe not only with the technology, they maybe were more impressed with our openness.”

Having little experience with western attitudes towards intellectual property and non-disclosure agreements, they presented a stark contrast to other companies with which Apple must have been used to dealing. Pachikov explained: “We didn’t ask them to sign anything, we were extremely open, because we were naive and eager.”

“We left Silicon Valley, we had our contract, and we received a check from Apple which we put in a vault in Russia. It was a $500,000 licensing fee. I returned to Moscow and the next day the coup happened in Moscow!”

“I remember Larry asked to have a private conversation with his staff for about 20 minutes. They left the room and they came back and told us that they would like to license our technology. Of course we hadn’t signed any contract, but they told us they’d like to license the technology.”

Big companies rarely act impulsively, even less so when money is involved, so Apple offered ParaGraph an evaluation agreement. Pachikov remembered: “They told us: ‘Would you agree if we develop the technology here and there. We would like to send our people to Moscow, see your people and facilities to be sure that you really exist.’ And we said yes. They offered us some price, about $75,000, to compensate us for our headache or hassle for evaluation.”

“Fine. $75,000 was a lot of money for us. We signed the evaluation agreement, and they sent a person, Rusha Bozinovic, to Moscow. He used to go to Moscow University and spoke Russian.” Apple liked what they saw and, after another trip to Silicon Valley, Apple decided to license the technology from ParaGraph.

Pachikov recalled: “At that time, our original founder, Scott Klososski, sold his portion of ParaGraph to another American investor, whose name is Ron Katz. Ron appointed one of his executives to be CEO of ParaGraph. His name is Bill Perlman and Bill was involved in the negotiations with Apple. Finally, we negotiated a deal and we signed the contract in July 17, 1991.”

Pachikov was satisfied with the contract. “It was a very good contract from a legal standpoint. There was no exclusivity for Apple, and there was a lot of up-front money. I hope it’s not extremely confidential and secret now but Apple paid us, together with the license fee, development, and porting, more than a million dollars up-front, plus royalties from future devices.”

“We left Silicon Valley, we had our contract, and we received a check from Apple which we put in a vault in Russia. It was a $500,000 licensing fee. I returned to Moscow and the next day the coup happened in Moscow!”

A Tense Period

It seems almost amusing now, as Pachikov described the turmoil, however, it was anything but that. “My wife woke up, and my brother called me to switch on the TV. I switched on the TV and my first reaction was, ‘poor Larry, he will be worried, his hair will turn grey.’ He signed this contract, and there is once again communism or some struggle.”

Pachikov continued: “I came to the office and I started to call everyone in the US from Moscow and told them, ‘don’t worry, the problem is only for three days.’ Honestly speaking, I wasn’t so optimistic. I believed that it might be a couple of weeks. But I told everybody three days. All our programmers were in the barricade near the White House, but I just said not to worry.”

Part of the Apple Team

Throughout the conversation, Pachikov stressed his feeling of loyalty to Apple and, because of this, felt justified in offering useful criticism of his partner. “This is not even a criticism, it’s just reality: nobody trusted any of our opinions beyond the technology of recognition. If I opened my mouth to tell anybody my opinion about user interfaces, or recognition, nobody listened. The attitude was ‘you’re Russian, you understand little.'”

In Pachikov’s opinion, this is what led Apple to make some important mistakes in the original Newton. “They put us in a situation of developing technology absolutely blind. I mean we didn’t know what we were developing for. They sent us specifications, they checked our demos, and they told us what they needed.”

He continued: “But you see, when you develop something and don’t know who or how people will use it, your engineering decisions must be much wider. You see, we didn’t know anything about the PDA. I never even saw a description.”

He concluded: “It’s not surprising, I think it’s Apple’s culture. They have some paranoia about confidentiality and secrets. This is why we developed all of our technology hearing zero about how it would be used, no information about the operating system, no information about applications, nothing! Of course some rumors were going around.”

Pachikov did have his suspicions about the architecture. “We happened to think that it might be a pen-based device, maybe something small, maybe with an operating system. That’s all. Finally, after one or two years of working together, people at Apple completely changed their attitude, and just recently a few months ago they started to communicate with us, and to listen to our suggestions about the user interface.”

“After we saw the Newton, and after we got the Newton, we went to them and told them that a lot of problems could have been avoided if we had known they were going to do this, we could tell them that it was wrong.”

As an example, Pachikov recounted the now-classic problem of entering a name that is not in the dictionary. He noted: “It was very strange because in our system, it is very easy to put it in a mode for character-by-character recognition, with no dictionary. But they didn’t do it, they didn’t listen to us.”

In addition, like all other recognizers, ParaGraph’s system can often make mistakes based on incorrect word segmentation. Pachikov explained this by saying: “When people write on the Newton, they start to write a sentence and they have space in the beginning. But at the end of the screen everything’s tight. The system doesn’t know that; the system actually connects, groups together, the words at the end of the screen.”

“We had a very trivial idea about how to have a small user interface solution to correct these mistakes, to tell the system it has made a mistake with word segmentation, to stop there, split words and recognize again. I know a lot of these improvements will be done on the next version. I hope Apple doesn’t sue me for telling that the next version is so much better, that it’s even difficult to imagine. It is significantly faster, more reliable, and everything.”

Meeting Expectations

Like many in the industry, Pachikov lamented about people’s expectations from handwriting recognition systems. “Don’t forget that the Newton has a powerful processor, a lot of memory, but it is still a PDA. It does not have 16 MB or 8 MB, it’s not a Pentium processor, it’s not a Sun. And digitizers also have limitations.”

In fact, Pachikov feels that the industry may have approached pen computing from the wrong end. He explained: “I had a discussion with Bill Gates a couple of years ago about pen computing. I said to him that the industry has made a mistake starting to develop PDAs and bringing pen computing to small devices. The industry made a mistake starting to bring to the market pen computing for consumer products, like the Newton, or like his WinPad.”

“I said to him that if I was the one who made these decisions, I’d prefer to start pen computing, developing recognition and applications, for powerful workstations, such as Suns, for editors. This is a vertical market, and they have no sensitivity to the price. If the recognizer cost $2000 or $3000, for these people it’s not very important. And they have no limitation with the processor or memory. If recognition technology and pen-computing were accepted by this professional group, it could go down and down, and people would know what to expect.”

Pachikov summarized by saying: “You see, when you take a Newton and it doesn’t work, or doesn’t work very well, but you saw it work on your workstation, then it’s fine because it is a small machine with smaller memory, so you know what to expect.”

“But with recognition technology, they can only compare this recognition with another recognizer in their brain. And they know that they can recognize very easily. My mother, who is 70 years old, has never seen computers except maybe a few times.”

“When she asked me what we were doing with our software, I said we are developing recognition technology, and I gave her a demo. I wrote something and had it recognized. She wasn’t impressed. She was very impressed with the spreadsheet demo that I gave her. I wrote a couple of numbers and it was automatically calculated, everything changed. But when I wrote my name, she wasn’t surprised because she can also do it.”

“The people at Palm Computing know human psychology better, and they understand that people will agree to write differently. As a marketing business decision it’s great, but there’s no technology. There’s no reason to compare it with this technology.”

Marketing Recognition

Pachikov also wanted to set the record straight about Palm Computing’s Graffiti. “Everybody tells us that Graffiti can recognize 100%. First of all I like the technology of this company very much. I’m sure that it’s good for us, because the more applications and more recognition systems there are for the Newton, the more the Newton sells.”

“The people at Palm Computing know human psychology better, and they understand that people will agree to write differently. As a marketing business decision it’s great, but there’s no technology. There’s no reason to compare it with this technology. It’s not fair, like a lot of people who wrote in their articles, to compare the Graffiti recognizer as a technology, as a breakthrough in character recognition.”

Pachikov also described how ParaGraph created the compression technology inside the Newton. He noted the strange irony of our industry that people were generally unfamiliar with this fact, mostly because the system worked without problems.

Back in the USSR (Russia Really)

ParaGraph continues to work hard on the next generation of software for the Newton, but also has a range of other projects under way including a virtual reality-like application for visiting historic and fictitious worlds. The company is also working to expand their technology to other platforms including small, wireless communication devices.

ParaGraph has a staff of 40 in Sunnyvale, California—not far from their Cupertino partners—and about 100 back in Moscow. The higher number in Moscow is partly accounted for by bodyguards, cleaning people, and others. Pachikov explained this by describing the need for self-sufficiency in Russia.

When asked if the guards are to protect against Russian organized crime, he replied: “Of course.”