Feb 20, 2013

Challenge: The software engineer genuinely didn’t understand it was wrong to use the software.
Challenge: HR wasn't fully aware of the risks when hiring.
Challenge: The IT department grew too fast and had poor safeguards.
Challenge: The firm had no formal process in place for this sort of crisis.
A final word from Capsicum
Meet Capsicum Consultant Alfonso Salgado

"I would never use stolen software!" Bogdan Lupescu’s face was red, his voice hoarse. The Alargent Capital IT staff shrugged and continued packing up his belongings. "Every line of that software is mine; I swear!"

Down the hall Fred Collins, the firm’s chief counsel, gathered key staffers in a conference room. "Have we isolated all of the stolen code?" The CIO, Tim Laine looked stressed. "We’re scanning every laptop, every portable drive, but I doubt we got all of it. It could be on a dozen USB thumb drives…"

"Or email, or CDs," Serena LaVette, the VP of networking said, sketching the data flow on a white board. "Stolen data is like an oil spill in the ocean. The data just goes… everywhere."

Fred grimaced. "Well, we better bring in somebody fast before…"

As if on cue, Chief Operating Officer Frank Cardeza burst into the room. "The lawyers from the company where Lupescu used to work just called. They’re suing Lupescu – and us, too! They put a distinctive dummy function in Lupescu’s code and can prove we’re using stolen software."

Alargent Capital is a fictional hedge fund. But the issue of stolen software is all too real – and in today’s hyper-connected world, stolen software can go global in milliseconds and cost millions.

How does something like this happen?

Challenge: The software engineer genuinely didn’t understand it was wrong to use the software.

Bogdan Lupescu was a brilliant software engineer who recently emigrated from Romania. Alargent had been trying to hire him for more than a year. However, different cultures sometimes have different understandings about what’s theirs and what belongs to the company. In the culture the engineer came from, if you wrote the software, you were free to use it, whenever and wherever you went. For Bogdan, it was his idea and so it was his software – he didn’t understand that in the U.S., these rights belonged to his former employer.

Solution: Everyone in the hiring process needs to be cognizant of cultural differences. Do not make the mistake of assuming that “everyone knows” or that questions of ownership are universally understood. Always err on the side of making the legal and cultural norms explicit.

Challenge: HR wasn't fully aware of the risks when hiring.

All employment agreements Alargent had new employees sign contained warnings about using software from former employers. Legally, the firm was covered. However, as a practical matter, this warning was essentially invisible. During the on-boarding process, new employees were required to sign more than a half-dozen documents covering everything from health care benefits to retirement plans. In truth, few, if any new employees ever read through the documents carefully. What's more, the caution about not using stolen software was buried in legal language in a three-page document.

Solution: In every interview – and especially interviews with IT staff – Human Resources needs to emphasize the point that employees are forbidden from using data or software from past jobs. Everyone in the interview process needs to echo this. Then, during the employee on-boarding process, this point needs to be reemphasized: "We don't take our competitors' secrets. We don't want them. If you see anyone using software that the company doesn't own, you must report it". Vigilance on this subject needs to be a part of the company's culture.

Capsicum can help you audit and improve your current processes to avoid future problems.

Challenge: The IT department grew too fast and had poor safeguards.

Alargent had experienced tremendous growth in a short time, and software was a significant factor in that growth. The CTO was young and one of the brightest people around. However, he lacked experience in setting up software development environments. He was extremely resourceful about finding great solutions quickly and inexpensively, and had a great eye for identifying up and coming talent. But day-to-day operations were not his strong point.

Solution: Consider hiring an experienced manager from a software company to partner with the CTO to create a more structured and disciplined technical environment. Rights and privileges are critically important. Ideally, segment duties and have multiple employees work on the software; more eyes mean fewer problems. Require engineers to log what code they have taken out, and what code they put back in. Ensure that developers and managers keep track of what is truly theirs and proprietary. Never allow a line of code for software to be written without strict versioning control.

Lastly, remember that prevention and discovery are closely related. Capsicum can help you set up an environment that adheres to best practices to protect the company.


Challenge: The firm had no formal process in place for this sort of crisis.

When the crisis hit, Alargent scrambled as best they could to address the issue. But things progressed so quickly that they were unable to keep up with changing circumstances. They forgot that digital safeguards aren’t enough. The day Bogdan Lupescu’s computer was taken to be examined, a disgruntled engineer who felt Bogdan was mistreated, returned after hours with a sledgehammer and destroyed the server rack.

Action Item: Immediately after a software security breach is discovered:

  • Gather as much information as you can. Bring in HR, engineers, and supervisors.
  • Determine who you can get the information from, and who you can trust.
  • Understand your programming and hosting environment. How is it set up? How is it backed up? Where are the archives?
  • What does the software do?
  • Outline possible negative scenarios. For example, if the offending engineer is angry, could they come back through a hidden VPN tunnel and delete or alter your source code?
  • Remember to secure your computing environment physically, as well.

Action Item: Prevent future breaches

  • Set up systems to monitor all of your software, to proactively check for problems, and to react and remediate if errors occur. You need to be able to know that something wrong has happened, and what to do about it.
  • Document any hybrid systems – a lot of software today is partly open source, and partly custom-coded. Know exactly what you own, and what you do not own.

If your company has been impacted by using stolen software – or if you find another company has stolen your software – it makes sense to bring in a mutually trusted third party company like Capsicum to analyze the problem and set up a plan for remediation. This third-party company will take possession of all the data, and take images in a way that is legally and forensically sound. Lastly, the company will help with remediation, to eliminate the stolen software within the offending company. Legal counsel from both companies will work closely with a third-party company like Capsicum to ensure that everyone’s interests are protected.

A final word from Capsicum

Security and processes to ensure that your company is not running stolen software can never be entirely foolproof. However, there are many time-tested, practical processes that you can establish to radically improve your defenses. A crisis like the one Alargent Capital faced in this case study is costly, distracting, and ultimately damaging to the business.

At Capsicum, we are highly experienced in handling this type of crisis. We can analyze the problem and set up a plan for remediation, to eliminate the stolen software within the offending company.

You can also work with Capsicum to establish protocols on the front-end to help establish and maintain a safe, well-maintained operational environment that is customized to your unique business – before problems present themselves.

Meet Capsicum Consultant Alfonso Salgado

Alfonso joined Capsicum Group in June 2012 as a Senior Digital Forensic Consultant.

His nickname is "Fonz", and he more than lives up to the level of cool you'd expect. In fact, he can do things that would make the Fonz from the old Happy Days TV show green with envy.

He did counter-intelligence in the army for 6 years and had Top Secret clearance. Later he conducted forensics for the DEA, FBI and state and local law enforcement with the NDIC, and then worked for Xerox Corporation. Fonz is fluent in Russian and Spanish. His enthusiasm for forensics is contagious. "The most fun part of the job is solving the puzzle. It's something new every time, there's always some sort of challenge. In the military, we called these surprising challenges 'alligators' -- you never know what's going to pop up. It's never boring!"

Alfonso vividly recalls his involvement in a drug case, where law enforcement officials in the Dominican Republic were working to take down a local drug kingpin and were able to obtain a great deal of data from computers and cellphones. The data was mostly useless, except... Alfonso noticed that certain addresses kept popping up. Curious, he dug in and discovered a high number of FedEx and UPS tracking labels. Using these, his team was able to identify these as addresses the drug lord was using for drug drop-offs. Local law enforcement were unwelcome guests at the next drop-off, and the drug lord was busted on the spot.

In an intellectual property case in which designs for a new jet engine were stolen from an engineering company, Alfonso’s background and experience again, proved vital. The thief was a Russian IT worker who was very savvy about covering his tracks -- he not only deleted the files of the engineering designs he stole, he had overwritten most of them. What he didn't count on was that Alfonso is fluent in Russian. After reading all of his emails in Russian, Alfonso was able to recover a few of the deleted files. The engineering company now had the smoking gun they needed.

While working on an identity theft case, it was nearly impossible to determine how certain personally-identifiable information was leaking out to criminals in California -- but it had to stop immediately. Deep forensic examination of emails and call records revealed that a local mob in Romania was recruiting people at call centers there to sell the information -- which was then re-sold to criminals in Southern California. Alfonso worked with local law enforcement both in Romania and the FBI in California to stop the leaks.

Fonz performs computer forensics out of our Dallas office. He's been married for two years and has a 5 month old daughter named Amelia. He's totally energized by being a Dad. As he puts it, "It's a whole new kind of happy". We're happy for him, and we're thrilled he's on our team.

NOTE: Details of the cases discussed here have been altered to maintain confidentiality.

Capsicum Group, LLC, is a technology and consulting company devoted to helping businesses improve operations and successfully complete technology-related projects. Its practice is focused on various disciplines including: digital forensics and investigations, data and tape recovery, electronic and paper discovery and technology security and compliance. Contact Capsicum today!