Stories‎ > ‎

A Year on a Cyber Protection Team

posted Jul 3, 2016, 5:44 AM by Michael Lenart   [ updated Sep 8, 2016, 2:41 PM ]
By Joe Marty

"Leerooooyyy Jenkiiiiins!" cried out our senior network analyst.

"Leerooooyyy Jenkiiiiins!" replied the rest of our team.

In heat of cyber battle, with every head ducked in their cubicle, we used the infamous World of Warcraft battle-cry to signal when Red Team activity is detected on the network we're protecting. This particular announcement was one of many heard from our team during Cyber Guard 2015 last year. Life on a cyber protection team (CPT) can become burdensome amid cyclical tasks and compliance checks, so we try to ignite motivation anytime we spot suspicious activity. It's worked well for us so far, so I think we'll continue to keep the mood light when we're on the keyboards.

I've been on an Army service CPT for two years now, and it has been an incredible, dynamic experience as the Army builds its capacity to defend cyberspace. I was away from my home station for 20 weeks in 2015, so I'd like to share some of my experiences over the course of that year. Our team supported two incident response missions, attended two collective training courses, and attended a joint cyber exercise. But before I pull back the curtain to expose what life is like on a CPT – what is a CPT?

Photo credit: electroniccommerce.com

There are four main types of CPTs: Combatant Command (CCMD), National, Department of Defense Information Network (DODIN), and Service CPTs. The CCMD teams are under operational control (OPCON) of the Geographic Combatant Commands (GCCs). National teams are OPCON to the Cyber National Mission Force (CNMF). DODIN teams are OPCON to Joint Forces Headquarters (JFHQ). Service CPTs are OPCON to their respective service commands; so our Service team receives missions from Army Cyber Command (ARCYBER).

Each team was originally organized into five squads: Mission Protection (MP), Discovery and Counter-Infiltration (DCI), Cyber Threat Emulation (CTE), Cyber Readiness (CR), and Cyber Support (CS). I say "was" because many teams are restructuring how they are organized because, although the "five-squad model" provides a functional approach to deliberate missions, it is inefficient for conducting incident response missions. I'll explain the difference in mission types soon... first, let me introduce you to the team.

Every unit has a leader. The CPT Chief is a Major (O-4) billet, but it is not uncommon for a Lieutenant Colonel (O-5) to lead a CPT. The Chief leads the team by providing direction, vision, and purpose; managing administrative work, long-term planning and coordinating all CPT events; and being a heat-shield and advocate for the team. The Chief relies on the non-commissioned officer in charge to ensure administrative deadlines are met, priorities of work are understood, and Soldiers on the team are professionally developed. The Chief also has a Department of the Army Civilian Operations Officer to help plan, de-conflict, and coordinate operations and events for the team. The final member of the headquarters element is the Cyber Warfare Planner; slotted as a chief warrant officer 3 (W-3), the senior warrant officer on the team assists the operations officer in planning and coordinating CPT missions, and fills in where required as the senior technical expert on the team.

The MP, DCI, CTE, CR, and CS squads are each managed by their own Computer Network Defense (CND) Manager; although the billet is for a Captain (O-3), it is common to find a Major managing one of the squads. In the five-squad model, each CND manager would supervise and manage seven personnel; however, the reorganized CPTs have some officers slotted as CND Managers, while they actually work more like a staff officer for the team. Whereas other CND Managers may serve as more of a platoon leader, albeit a very small platoon by traditional standards. These mini-platoons of 8-10 Soldiers are referred to by some as Cyber Response Teams (CRTs), and some CPTs assign a different focus area for each CRT. My particular team is mission-focused on securing Industrial Control Systems / Supervisory Control and Data Acquisition (ICS/SCADA) for critical infrastructure and key resources (CIKR) in Army enclaves. I know of another Army service CPT focused on securing personally identifiable information (PII) platforms and databases, and a few other mission-focus areas are still being considered for other CPTs.

Photo credit: www.stripes.com

Because CPTs are expected to maintain the ability to deploy a CRT in response to an incident, one CRT is usually "on call" as the quick reaction force (QRF) for the CPT, while the other CRT is committed to the specific mission focus of the CPT (e.g., ICS/SCADA security, PII databases). This division presents a challenge when it comes to manning and equipping for missions. The "fly-away kits" containing the tools needed for each type of mission must be configured with specific software – such as support to an incident response mission versus a deliberate assessment or clearing mission in an ICS/SCADA environment. Consequently, we keep pre-configured hard drives for each mission type on standby. To keep everyone on the team proficient in both incident response and the CPT's primary mission focus area, and to keep skills from atrophying, we rotate personnel among the different mission types.

Besides the technical skills (people) and tools (equipment), the other big difference between the mission types is the task organization. The five-squad model works for deploying in support of a deliberate mission: CR assesses compliance; MP assesses current security posture; DCI hunts for threat activity; CTE tests assessment findings and mitigation; and CS reinforces the other squads and trains the personnel in the supported organization. However, in an incident response mission our CPT reorganizes into our "battle formation" learned at the Advanced CPT Training – Pilot (ACT-P) course.

For 8-10 hours per day, 6 days per week, over 6 weeks, our team trained to conduct incident response using four cells: Coordinate, Monitor, Pursuit, and Harden. While having the Harden cell is useful during training and exercises, CPTs will very rarely be permitted to make any changes to an organization's network without following a strict change management process. In those cases, we integrate our Harden cell members into one of the other cells to provide recommendations to the organization when necessary. The Coordinate cell manages the Monitor and Pursuit efforts, tracks the battle, and interfaces with the organization and higher HQ. The Monitor cell conducts active network security monitoring (NSM) on a sensor grid passively tapped into the organization's network and managed on a non-reusable, out-of-band, private virtual local area network (VLAN). The Pursuit cell tracks down leads provided by the Monitor cell by investigating host systems using a variety of hunt methods.

This battle formation of Coordinate, Monitor, and Pursuit worked very well for us at the ACT-P course, so we stuck with it three weeks after graduation when we participated in Cyber Guard. A few notable differences between the ACT-P course and Cyber Guard were initially dismissed, but quickly posed serious roadblocks to our performance and required adaptation.

The first difference was the seating arrangement: at the ACT-P course we had a large room to ourselves with large wide-screen TVs to help track the battle, everybody was seated at open tables facing the coordinate cell at the front of the tactical operations center (TOC), and the only noise in the room was generated by the team. At Cyber Guard, our team was divided across three rows in an open bay shared among four other teams, the rows each faced opposite directions, and every computer was set in partitioned desks (i.e., cubicles).

At the ACT-P course, whenever the Monitor cell spotted some potentially (or definitely) malicious activity on the network, in a somewhat loud voice one of them would say "Attention in the TOC..." and explain what they spotted. This would prompt the Coordinate cell to determine the next move for the team, and would enable the Pursuit cell to prepare to shift their focus. At Cyber Guard, if one of us yelled "Attention in the TOC," it would either (a) not be heard, (b) silence the entire bay and disrupt everyone's operations, or (c) be complete ignored by everyone. So, we adapted our notification to make incident response more fun, and get our team's attention: "Leeerooooooyyy Jenkiiiiiiinsss!"

Photo credit: engadget.com

Just about every World of Warcraft player and YouTube addict recognizes that battle-cry. In homage to the crazy teammate of Internet fame who, with complete disregard for his team, tired of waiting on them to develop a plan, stated "Times up chumps, let's do this..." then he ran into battle alone and unafraid screaming his online persona’s name "Leerooooyyy Jenkiiiiins!" The phrase is recognizable, it lightens the mood, and it makes all the other teams around us think we are insane. It was perfect.

Whenever someone spotted anything strange or obviously malicious, they would loudly cry out " Leerooooyyy Jenkiiiiins!" and the rest of the team would echo the battle cry before gathering together to receive the report from the analyst. This helped everyone stay on the same page during fast-paced cyber operations in a shared, boisterous bay full of CPTs, distinguished visitors, evaluators, and contracted support. We also established the periodic leaders’ huddle every two hours to keep everyone current on each cell's activities and help the battle captain report updates to higher, direct cell activities, and effectively track the mission.

Cyber Guard turned out to be a great exercise of everything we learned at the ACT-P course. It helped us refine our processes, improve our communication, and build confidence in our abilities. It also prepared us for our real-world incident response mission that we would be sent on six weeks later, and it reinforced the feeling of how unprepared we were for our first mission last year.

In mid-January 2015, I attended the mid-planning conference (MPC) for Cyber Guard 15. Just six weeks prior, most of our team had completed the first "pipeline training" course all CPT members go through, the Alternative Introduction to Cyber Core. On January 1, 2015, our team was declared to have initial operational capability (IOC). At the end of the MPC my boss informed me that our team had been assigned a mission by ARCYBER, and we were needed at the Pentagon... immediately.

We were stoked! We had just reached IOC, and we're already going on a mission?!! This is going to be awesome! ...or, so we thought. By the time we arrived, another CPT had already "cleared the terrain" and handed us their findings. Our job was to supervise the mitigation process. We couldn't implement any changes, so we were basically in a consulting role managing the vulnerability remediation across departments. It was a CPT's worst nightmare: all the responsibility, and none of the authority to make or direct any changes.

Thankfully, we found a way to hand off that mission after about six weeks. We had about one month to recover before deploying to the ACT-P course for six weeks. So, by the end of the ACT-P course I had been TDY for 11 weeks since our team was declared IOC. Cyber Guard brought my TDY time up to a little over 13 weeks. Our operations tempo (OPTEMPO) had been far more intense than I expected, and we were only half-way through the calendar year!

During the few weeks between Cyber Guard and our next mission, I was lucky enough to get a seat in the SANS SEC 573 (Python for Penetration Testers) and SEC 560 (Penetration Testing and Ethical Hacking) courses with the warrant officer military occupational specialty (MOS) 255S class. Throughout the year, the Cyber Protection Brigade is offered remaining seats in SANS courses which available CPT members can attend. These SANS courses often come with Global Information Assurance Certification (GIAC) certifications (if you can pass the exam) to add a little credibility to your resume. As the CTE CND Manager, I felt it appropriate to attend these courses if the unit were offered seats. Luckily, my CTE interactive operator got us seats in the courses, and we ended up winning the capture the flag competition at the end of the Python for Pen-testing course.

After recovering from Cyber Guard late June through July, ARCYBER called on our CPT to respond to an incident in Washington, D.C. This mission was far different from our last trip to D.C., and was different for the other CPTs involved as well... this mission was truly global in scope. Several CPTs were deployed to locations across several time-zones to investigate the extent of an intrusion. The biggest difference in this mission for us, though: our skills were tested as we applied everything we learned in the ACT-P course, and practiced at Cyber Guard, to execute in a production environment. 

Photo Credit: U.S. Army

We were on flights to D.C. within 48 hours of mission notification. Within 6 hours of arrival, we had our equipment set up and were tapped into the network collecting and analyzing data. We divided up the forward element into a day shift and a night shift. Many of the details of the mission are classified, but what I can say is that we conducted 24-hour on-net operations for two weeks. After thoroughly investigating over two dozen servers and almost as many hosts suspected of compromise, and analyzing many terabytes of live network traffic, we reported our findings, briefed ARCYBER, then packed up to head home. It was a successful mission for us, and the classic mantra of the military was proven again: train as you fight, and fight as you've trained. We were battle-hardened in the cyber domain and ready for a break. We got about six weeks of recovery in garrison before we would start our next big event: Methodologies.

Before attending Methodologies with my team, however, I attended the two-week Army Cyberspace Operations Planners Course (ACOPC) in Maryland.  This is a higher level course meant to teach service members of all ranks, MOSs, and branches of service how to integrate cyberspace effects into land operations. As a field-grade officer with a master’s degree in Cyber Operations, the ACOPC was a condensed combination of what I learned in my graduate program and at the Command and General Staff College. But I think it provides a concise overview of how cyberspace effects can be a combat multiplier in land operations. Although I didn’t learn anything new, I appreciated being able to meet other service members, and to share my knowledge and experiences in the cyber domain. Returning from the ACOPC I headed straight into Methodologies with my team.

The "standard pipeline" (i.e., sequence of training) for CPT members is to follow the crawl-walk-run method of training individuals, then squads, then the whole CPT in a collective training event. Individuals are trained in Cyber Core Prep (CCP), Certified Ethical Hacker (CEH), and then Alternative Introduction to Cyber Core (Alt-ICC). Squads are trained on how they are expected to operate in Methodologies. CPTs are collectively trained at exercises like Cyber Knight, Cyber Flag, Cyber Guard, and Red Flag. The scheduling and space availability of the pipeline courses, and the irregular arrival of new personnel, prevent entire teams from completing all training together, and in sequence. Most of our team attended Alt-ICC together at the end of 2014, but we couldn't attend Methodologies until the fall of 2015 - almost a year later! Fortunately, we were at least able to attend the ACT-P course to prepare us for Cyber Guard and real-world operations.

In Methodologies the team is separated into different rooms for CTE, DCI, and MP. (To review, these stand for Cyber Threat Emulation, Discovery and Counter-Infiltration, and Mission Protection, respectively.) There currently is no Methodologies for the Cyber Readiness or Cyber Support squads, so these team members fill open seats in one of the other courses. The priority for filling is often MP or DCI, then CTE. Mission Protection is often considered the "primary task" of a CPT; however, DCI tends to be more fun and interesting to cyber operators, so those seats are often filled first. As an ethical hacker, I had a blast in the CTE Methodologies course.

In the final week of Methodologies, squads are brought together for a capstone followed by a team vs. team capture the flag competition. The capstone requires each squad to do what they were trained to do in their respective courses. MP conducts vulnerability assessments and develops the risk mitigation plan, DCI hunts for adversarial presence in the host systems, and CTE emulates the threat by attempting to penetrate and laterally move through the network, escalate privileges, and exfiltrate data. Each team develops and delivers an out-brief and Risk Mitigation Plan (RMP) to the organization to conclude the capstone exercise. The capture the flag competition requires the team to defend its systems while attacking the other team's systems. It's fun if you have skilled offensive and defensive operators on your team (which we did).

Shortly after completing Methodologies, ARCYBER assigned us our next mission: ICS/SCADA security. This new realm for IT defenders is all the rage, and it requires much more specific training, tools, and TTPs to tackle successfully. Our glaring knowledge gap in conducting DCO on ICS/SCADA systems prompted the Cyber Protection Brigade to fund training for our team. All available personnel on our CPT attended the SANS ICS 410 course (ICS/SCADA Security Essentials), then the ICS 515 course (Active Defense and Incident Response) one week later. Earning the GIAC Industrial Control Systems Professional (GICSP) certification by passing the ICS 410 exam adds a little credibility and confidence to our team, but the real benefit was the knowledge gained from the SANS instructors. 

Photo credit: U.S. Army Cyber Protection Brigade

During the week between SANS courses, I traveled to my alma mater, the University of North Georgia (UNG), for their Branch Day to recruit talent by encouraging them to commission into the Cyber Branch. It was a great turnout, and I enjoyed talking to future cyber leaders. I also use one of our days off to be a guest speaker at the local middle school. Engaging the community on cyber topics keeps you grounded, and it is a great reminder of why I am in this field: most users are oblivious to security.

One week after finishing our second SANS course, we conducted a leader's recon of our first mission location. The purpose of the recon was to establish our relationship with the organization, determine the mission requirements to complete a survey and cyber security assessment, and gather as many artifacts and documentation as we could. After a week of touring most of their facilities, taking photos, and asking tons of questions, we headed back to home station to conduct Intelligence Preparation of the Battlespace (IPB) and prepare our equipment and personnel to conduct the survey. 

Just 10 days after returning from the leader's recon, the CPT Chief and I flew to Idaho National Laboratories (INL) to learn more about the Industrial Control Systems-Computer Emergency Response Team (ICS-CERT) and how they can enable our DCO on ICS/SCADA systems. The quality of professionals working there is impressive, and they briefly demonstrated a few of the cutting-edge projects they work on that are truly astonishing.  After touring INL, it's no wonder every person I encountered had been working there for over 20 years! Hopefully we can include them on our future ICS/SCADA missions.

That brings the mission travel tally up to about 20 weeks at the end of 2015. By this point we were just beginning our ICS/SCADA security mission. Since January of this year I've been away from home five weeks, and I've been just as busy helping plan, coordinate, and manage our ICS/SCADA mission. In high school I knew I would become an Army officer, and around that time my father informed me that the Army had jobs where "you get paid to hack government systems to help them improve their security." After 12 years of service, I'm finally doing what I set out to do back in JROTC: hacking for the Army, and loving every minute of it.


About the Author


Joe Marty is the Cyber Threat Emulation CND Manager on the 154 Cyber Protection Team in the U.S. Army Cyber Protection Brigade at Fort Gordon, GA. He is a field-grade officer in the Cyber branch and has served in the US Army for 12 years. He holds a master's degree in Cyber Operations from the Air Force Institute of Technology (AFIT), a bachelor's degree in computer science from North Georgia College and State University, and five professional certifications (CISSP, CEH, GPEN, GICSP, GPYC). When he's not traveling with his team, Joe enjoys writing and traveling with his wife and two children.