Authors: Dilshad Mustafa
In the weekly
status meeting, Nitesh from Dochamk Bank joined the call. Nitesh worked as an
IT Manager. As usual, Satish walked through the Excel sheet containing the
status of each activity. Whenever anybody asked any technical questions, he
would ask either Vinay or Himesh to answer. He would then ask Vinay to explain
the work on the POC.
And as usual, Vinay
observed Murali dropped off from the call after the first ten minutes of the
meeting. Vinay knew Murali gave some petty reasons like “I have to join another
meeting now”, “I have to attend an audit discussion”, “I have a meeting with
the IS team for infrastructure setup”, “I have to attend a project management report
meeting” the list went on and on. Murali was very clever to schedule such
meetings to overlap with the weekly status meeting. He would always cook up
something. Vinay thought he was very good at that.
Vinay and
Himesh would then explain the POC work and what work they were doing then.
Nitesh asked
questions like, “What is your achievement this week? How are you planning to go
about closing the gap for the transaction rate?”
“I don’t see
any value-add here. You guys should take up some extra work and do something
extra. I mean you should take steps to walk that extra mile,” said Nitesh.
Nitesh said,
“Vinay, I see everything is progressing in a casual manner. The team is working
in a very relaxed mode. I see they are very relaxing. I’m not satisfied with
the progress so far. I’m quite disappointed with the results. We have fifteen
days left to complete the POC and you are saying the best we have is forty TPS.
You should be knowing if we don’t get at least two hundred TPS, this project
will get scrapped.”
“I don’t see
the team picking up pace or understand the tight timelines we have. I would
like to see some solid results next week,” said Nitesh.
Murali was
listening to all this from his phone in his cubicle. Though he had said he was
dropping off the call to attend another meeting, he actually didn’t hang up the
phone or dial in to another conference call. He had put his phone on mute and
he had a head phone plugged into the phone. He was listening to each word
Nitesh was saying.
Satish said,
“Yes Nitesh. Vinay and Himesh will update you regularly about the progress.”
“Ya sure we
will do that,” said Himesh.
Himesh
realized that no matter how hard he and Vinay worked, the extent of their
efforts would not matter much to anyone and even less so to Nitesh and Anna.
“Why don’t they
just cut this SMX configuration out and concentrate on only EMG? We will have
time to at least try out the three approaches,” said Himesh to Vinay while they
were discussing in private.
“If only if
they realize how hard it is for us to build for two platforms at the same time,”
said Himesh.
Vinay slowly
realized Himesh was biased towards EMG configuration.
“I’m going to
use hardwired technique and rebuild the SMX POC. It will raise the TPS to at
least hundred TPS,” said Vinay.
“What’s
hardwired technique?” asked Himesh.
“It’s a way of
writing spaghetti-like code. You reduce use of arrays and function calls in
code. You will get one big monolithic code. It will be slightly faster,” said
Vinay.
“Why are you
doing that for SMX? Why don’t you do that for EMG?” asked Himesh. His face showed
concern and disappointment that Vinay wanted to try out things with SMX and not
EMG.
“It’s clean to
do that in SMX. SMX is compact and highly structured. EMG has so many features.
It has become a very big system. I feel it’s not efficient in handling certain
things…it’s kind of like bloated software…slow, you know what I mean,” said
Vinay.
“TeraSMX is a
monopoly. It uses very proprietary hardware. PicoEMG uses hardware based on
Open Architecture Set. If you try out things only on SMX configuration used in
TeraSMX hardware, it’s not fair for EMG configuration used in PicoEMG hardware,”
protested Himesh.
“I see so you
want EMG and PicoEMG hardware to win the TPS contest. You can be frank with me,”
said Vinay.
Vinay used
hardwired technique and rebuilt both POC configurations. EMG now yielded eighty
TPS and SMX gave one hundred and twenty TPS.
Only ten days
were left to complete the POC.
Vinay worked
over fifteen hours over the next few days. During night he would think about
using new techniques like equivalence partitioning, orthogonal sets and using
multiple instances of back-end systems. During day time, he would apply those
techniques and rebuild both POCs for SMX and EMG configurations and document
the results. Sometimes he applied a combination of techniques and analyzed the
results.
At tea break,
Himesh said, “The client thinks five resources are working on the POC but only two
of us are doing the actual project work. We are working very hard and sweating
it out. These guys Shanthi, Murali and Satish are enjoying.”
“Can we clock
the overtime hours and the hours we have worked on Saturdays and Sundays while
filling the timesheet?” asked Vinay.
Employees had
to fill in the timesheet every week for the days they had worked excluding
Saturdays, Sundays and company holidays. They could fill up to a maximum of nine
hours for each working day. So if someone worked for fifteen hours on a
particular day, he should still clock nine hours only while filling the timesheet.
“No. We can’t.
We are supposed to bill only up to nine hours every working day. And that
doesn’t include Saturdays and Sundays. If you do fill you overtime hours, your
timesheet will get rejected. So don’t do that. It’s a big hassle if it gets
rejected and getting it re-enabled again,” said Himesh.
“I see.
Official working time is from 9 to 6.30. But I see many people working from 9
to 9,” said Vinay.
“Yes. That’s
the unofficial working time here,” said Himesh.
Vinay liked
the simplicity and compactness of SMX. But he hesitated about the hardware
configuration and from what he had heard of TeraSMX’s monopolistic history and
people’s aversion towards it. People had told him stories like how they would
start dictating prices of annual maintenance contracts and increase prices at
their whim. He had heard they would sweeten the deal initially to get the bid
but once you got locked in with them they would slowly start increasing the
prices of everything. This was also one of the reasons why Dochamk Bank wanted
to move to PicoEMG hardware configuration from their existing legacy technology
based systems which was maintained by TeraSMX.
Knowing this, TeraSMX
had placed a counter bid with their flagship hardware configuration and
announced substantial discounts for a package deal if the bid went through the
evaluation process. The deal was so attractive that Dochamk Bank included TeraSMX
also into the evaluation process.
Vinay knew
people broadly fell under one of the two groups one was the Legacy Systems group
and the other Open Technology Systems group within Dochamk Bank. He also knew
he would get very little support from Dochamk Bank’s existing Legacy Systems
team as everyone knew what Holtezent team was working on especially Vinay and
Himesh. Vinay heard from Himesh the people in the Legacy Systems team thought
their jobs would be taken over by Holtezent’s team once this project got
completed and became operational.
In the
afternoon, Vinay and Himesh took a walk inside the server room. The room was
adjacent to the ODC and it required door access. Both Vinay and Himesh had the
door access. PicoEMG and TeraSMX had given their hardware free for exactly one
month for this POC. They hoped to bag this multi-million dollar hardware
infrastructure required if the project went ahead after the POC phase succeeded
in the feasibility study.
Vinay saw the four
PicoEMG’s hardware. It was bulky and bluish-grey in color. There were large
vents in the front, back and sides. It had four racks inserted into four slots
in the front of the machine. Each rack had four quad core CPUs. The EMG6500 was
a fierce beast of a machine.
“Why four
machines from PicoEMG?” asked Vinay.
“Don’t know
exactly. It may be two for running Xorbiz and two for running PicoEMG’s
database. It is up to us to decide how we want to use them,” replied Himesh.
Vinay started
to have some vague ideas how best to use these four machines.
They then went
near the TeraSMX hardware. It looked like a big wardrobe painted fully in
grey-black color. It was just one single machine with eight slots. Four slots
had racks inserted in to it and the remaining slots were empty. Each rack had
four octo-core CPUs. The SMX7500 from TeraSMX was a giant mammoth of a machine.
Vinay had
experienced earlier that the TeraSMX hardware was almost instant in its
response for any program he ran in it. He didn’t feel the same about PicoEMG
hardware. His logic told him four machines running in parallel processing
should be able to beat a single machine but in reality they couldn’t stand
against SMX processing speed.
Vinay and
Himesh discussed about this for some time.
“Something we
are lacking. I strongly feel a single machine no matter how many racks and CPUs
it had will reach its bottleneck earlier than what four machines could do. There
are various points where a bottle neck could occur, a network port for
instance. A single network port in a single machine will have less bandwidth
than four network ports in four machines. Also I doubt to what degree these software
are able to utilize multi-core CPUs. There is something we are missing. We need
to find out how best to use these four machines,” said Vinay.
“We cannot
achieve hundred percent parallel processing with these four as most of the
times the program running in any of the four would be waiting in queue to
access a common variable or resource for reading or writing a value. I do agree
concurrently running programs in these four should at least match if not more
than the speed of SMX. But comparing one-on-one, yes they are no match for SMX,”
said Himesh.
Vinay and
Himesh then discussed about possible approaches that could be used. An idea was
starting to form slowly how to concurrently run the programs in the four PicoEMG’s
hardware.
The highest EMG
could achieve stood at ninety TPS while SMX stood at one hundred and forty TPS.
Only five days
were left to complete the POC.
“Do something,”
pleaded Himesh. “I don’t want to work with SMX as back-end. Please do
something. Everyone here wants EMG to win the POC contest. It’s not me alone,”
said Himesh.
Vinay was well
aware people from his team preferred EMG. He decided he could not look at
things objectively. Vinay thought people had biases. They tend to like certain
things over others.
Vinay went in
to deep thinking. He was running out of time and many options to try out. He
simply didn’t have time to implement and analyze those approaches. He had to
carefully weigh in and select one or two approach which had the most
probability to yield the desired transaction rate.
This
effectively meant he had to gamble. And if he failed, there won’t be a project
to work on after this month.
Vinay
discussed this with Himesh and told him the risk involved with the gamble. The
approach he chose meant that they would not have any time left to try out the
other approaches.
“Whatever you
do you have my full support,” said Himesh.
Vinay then
explained the approach.
“We will create
a transaction balancer and feed the output to multiple equivalence partitioned
sets running in multiple instances. This will increase parallel computing of
transactions and improve TPS count especially with the four machines
configuration. This approach should also show some improvement with the SMX
configuration,” said Vinay.
Then they both
started working on it day and night. They would leave office at 11.00 at night
and come back and resume work at 7.00 in the next day morning. Sometimes they
would leave office at night 1.00 AM. They booked a cab whenever they had to
leave very late at night as the last bus was at 11.50 PM.
The last reading
for EMG stood at one hundred and fifty TPS while SMX stood at one hundred and
sixty TPS.
POC analysis
results had to be submitted that day to the customer. Himesh wondered what they
would do. They had not yet achieved two hundred TPS. Also, the results they had
till then showed TeraSMX as the winner, much to Himesh’s dismay.
Vinay
explained to Himesh there was one incorrect way of doing things with back-end.
Turning off index in a back-end system was not a proper way of creating a
back-end system. While it might seem faster for certain types of transactions,
it would actually slow down other types of transactions. Vinay said that by
turning off index in EMG configuration, it could easily reach two hundred TPS for
the kind of transactions chosen for this POC. But he cautioned against doing
that. He called it as the half-baked approach.
Vinay advised
that doing so would not be a proper way of designing the system and would not
result in an unbiased objective analysis.
Indexing was
done by creating a hash of the values of a particular data. For example, the
values of consumer name, that was the Bank’s customer name, could be indexed to
provide faster access for queries when searching by consumer name. A hash was a
thirty two character long string generated out of any text using a hashing
algorithm. While using index may make some queries faster, it might slowdown
other queries which did not use the indexed field to search or update owing to
the extra burden in creating the index by running the hashing algorithm for
every query.
“Everything is
about trade off man. You will get something, you will lose something. One
cannot get everything. Even if we have index also, certain queries which
doesn’t use indexed data will actually slow down. It’s all about trade off
which queries have higher priority and which ones have lower. It doesn’t mean
one way is right or other way is wrong,” said Himesh.
Himesh then
asked for the detailed steps from Vinay for doing those changes and went ahead
with the rebuild for EMG configuration. He was finally relieved when the
results for PicoEMG showed two hundred TPS versus TeraSMX’s one hundred and
sixty TPS. Apparently, he didn’t do the same for TeraSMX configuration to make
its TPS appear lower than PicoEMG.
Murali told
them to prepare a detailed power point presentation aka PPT. He told to start
the slides with background introduction, followed by chronology of events with
exact dates, followed by details of various approaches used with findings and
finally the results. He also told to add a section on Value-adds and Cost-savings
brought to Dochamk Bank and the accomplishments of Holtezent team.
During all
those days of breakthrough by Vinay and Himesh, Murali never missed a chance to
take credit for their work. Whenever a breakthrough was achieved, Murali would
send out a mail to Anna, Nitesh, Raghu and prominent figures in higher
management and Business Units within Holtezent and trumpeted about the
breakthrough. He would make sure to delete any references to the names of Vinay
and Himesh in the documents and emails. He didn’t include them in the list of
email recipients as well.
Satish then
met them and told them to put as much technical details as possible in those
slides detailing the various approaches. He told to list out all the challenges
faced and how it was addressed. He told, “Use Visio and create lot of diagrams.
PPT should look impressive. Also put detailed design diagram for the finalized
approach.”
Vinay knew Satish
and Shanthi did nothing in all those days of hard work by him and Himesh.
Satish was very busy with his day trading work buying and selling company
shares every day using a share trading online portal. He used office time and
office resources to carry out his day trading activity silently without anyone
noticing him. Vinay was well aware of what Satish was doing.
Shanthi did
the bare minimum of work possible usually spending three days on documenting
one use case which could have been done in just one hour. She was very good in
pretending to be busy at work. Himesh’s observing eyes had noted how she worked
as well. Her monitor screen would usually show one page from a document from
the morning to till she leaves office. She would scroll down the page a bit
every half an hour or whenever someone walked near her seat.
The next day
everyone dialed into the bridge for the conference call. Greenspan, Anna and
Nitesh joined the call from Dochamk Bank. Greenspan was the Program Director
for this Business Unit in Dochamk Bank, Anna was the Project Director and
Nitesh was the IT Manager.
From Holtezent,
Raghu, Murali and Satish dialed in to the bridge from their cubicle seats
separately. Raghu was the Project Manager from Holtezent. The rest of the team
members pulled their seats near Satish and listened to the call.
Everyone
joined the screen sharing session from their PCs and laptops.
Raghu started
talking. He first greeted the client and then introduced the people in the
call.
This was the
first time Vinay heard Raghu’s voice since he joined Holtezent. He had not yet
seen Raghu in person.
Raghu then
gave a background of the engagement. He talked much about the POC challenges
and accomplishments of Holtezent. He went over the chronology of events in the
PPT. He then moved on to Value-add and Cost-savings section. He boasted how Holtezent
completed a three months of work in the POC within one month time frame. He
talked about the industry standard of hundred TPS and Holtezent’s
accomplishment of two hundred TPS.
Greenspan
said, “I appreciate the results so far. Good leadership by Raghu and Anna. I
appreciate the team for giving a quick turn around on this. I look forward to
take this engagement to the next level.”
Murali then
went on to present the various approaches they implemented. He talked about
the technical challenges and how they overcome those. He then explained the
final approach laid out in the PPT and walked through the diagrams.