Cloud Logging and Monitoring: Security Fundamentals

Cloud logging and monitoring is the practice of capturing every control plane and data plane action in your cloud environment, storing those records securely, and turning them into alerts that a human or automated system can act on. Without it, you cannot answer the two questions every security incident starts with: what happened, and who did it.

Most cloud breaches are not discovered by a clever detection engine. They are discovered when someone goes looking through logs that happened to be turned on. That is the whole argument for treating logging as a day one control rather than a day ninety cleanup task.

Why logging is a security control, not an ops afterthought

Engineering teams usually turn on logs for troubleshooting. Security teams need something different. The difference matters:

• Operational logging answers "why is this service slow?" and tends to be verbose, short-lived, and application-specific.
• Security logging answers "who changed this IAM policy at 2am from an unfamiliar IP?" and needs to be tamper-resistant, long-lived, and centrally collected.

If you only have operational logs, your incident response starts from zero. If you only have security logs, you cannot reconstruct what an attacker actually touched. Mature environments collect both and route them to different destinations with different retention.

There is also a hard governance reason. Frameworks such as the NIST Cybersecurity Framework treat detection as a core function alongside protection. Auditors in almost every regime will ask you to demonstrate that privileged actions are logged and reviewed.

The three log types you need everywhere

Regardless of which cloud you use, security-relevant logs fall into three buckets.

1. Control plane and audit logs

These record API calls against the cloud provider itself: creating a user, changing a firewall rule, deleting a storage bucket, attaching a role. They are the single most valuable source for detecting account compromise and insider misuse, because almost every attack against a cloud account eventually shows up here.

2. Data plane and resource access logs

These record operations on data inside a resource: reading an object from storage, invoking a function, querying a database. They are higher volume and usually cost money to collect, so most organizations enable them selectively for sensitive resources rather than everywhere.

3. Network and workload telemetry

Flow logs, DNS query logs, load balancer logs, and host or container logs. These fill in the picture of lateral movement and exfiltration once you know an identity was compromised.

How the three major clouds implement this

Capability	AWS	Azure	Google Cloud
Control plane audit log	CloudTrail management events	Azure activity log	Admin Activity audit logs
Data access log	CloudTrail data events (opt-in, billed)	Resource logs via diagnostic settings	Data Access audit logs (mostly opt-in)
Network flow visibility	VPC Flow Logs	NSG flow logs and virtual network flow logs	VPC Flow Logs
Central log store	CloudWatch Logs, S3	Log Analytics workspace	Cloud Logging log buckets
Native SIEM	Security Hub plus partner SIEM	Microsoft Sentinel	Security Command Center plus partner SIEM
Managed threat detection	GuardDuty	Microsoft Defender for Cloud	Security Command Center threat detection

A few provider specifics are worth memorizing because they show up on exams and in real architecture reviews.

AWS. CloudTrail logs management events by default, but trails and event data stores do not log data events by default, and data events carry additional charges. That default is the reason so many S3 object-level investigations hit a dead end. See the AWS CloudTrail documentation for the full event taxonomy.

Azure. The activity log is retained for 90 days by default and then deleted. To keep it longer, or to route it anywhere useful, you create a diagnostic setting that exports it to a Log Analytics workspace, storage account, or event hub. Microsoft documents this in the diagnostic settings guide. Microsoft Sentinel then sits on top of the Log Analytics workspace as the SIEM layer.

Google Cloud. Admin Activity audit logs land in the _Required log bucket and are retained for 400 days, and that retention cannot be changed. Data Access audit logs generally must be enabled explicitly and, along with Policy Denied logs, land in the _Default bucket with 30 day retention unless you configure a custom bucket. Custom retention can be set between 1 and 3650 days.

The pattern across all three is the same: the cheap, low-volume administrative log is on by default with modest retention, and the expensive, high-volume data log is off by default. Attackers know this.

Building a logging architecture that actually works

A defensible design usually follows five steps.

1. Turn on organization-wide audit logging first. Use an organization trail in AWS, subscription-level diagnostic settings in Azure, or organization-level log sinks in Google Cloud. Per-account configuration drifts and gets missed.
2. Send logs out of the account that produces them. If an attacker owns the workload account, they should not be able to delete the evidence. Centralize into a dedicated logging account or project with tightly restricted write-only access.
3. Protect the log store itself. Enable object versioning and object lock or the equivalent immutability control, encrypt at rest with a customer-managed key where policy requires it, and alert on any attempt to disable logging.
4. Select data plane logging deliberately. Enable it for buckets holding regulated data, for secrets stores, and for database services, rather than turning it on globally and then panicking at the bill.
5. Define retention by purpose. Hot, searchable retention of 30 to 90 days supports investigation. Cold archival retention of one to seven years supports compliance and late-discovered breaches. Storing everything hot is the most common way teams blow a cloud security budget.

From logs to detection

Collecting logs is necessary but not sufficient. Detection means writing rules and baselines against those logs. Reliable, high-signal starting detections include:

• Root or global administrator account usage
• Multi-factor authentication disabled or an authentication method changed on a privileged identity
• Creation of new access keys, service account keys, or long-lived credentials
• IAM policy or role assignment changes that grant broad permissions
• Logging, monitoring, or threat detection services being disabled
• Public exposure of a storage bucket, database, or management port
• Sign-in from an unusual geography, ASN, or user agent for a privileged principal
• Large volume data reads from a resource that normally sees low read traffic

Managed services accelerate this significantly. AWS GuardDuty analyzes CloudTrail, VPC flow, and DNS data without you building the pipeline. Microsoft Defender for Cloud and Microsoft Sentinel play the same role in Azure, which we cover in our Azure security fundamentals guide. Google Cloud Security Command Center does the equivalent in GCP.

Use the managed detections as a baseline, then write custom rules for the handful of things unique to your environment. Nobody else knows which three service accounts should never touch your production database.

Common mistakes that defeat the whole exercise

• Logs collected, nobody reading. An alert that routes to an unmonitored inbox is not a detection.
• Noisy rules that get muted. A rule firing 400 times a day gets ignored within a week. Tune before you deploy.
• No time synchronization discipline. Correlating across sources is painful when timestamps disagree. Standardize on UTC everywhere.
• Retention shorter than dwell time. If your logs expire in 30 days and an intrusion went undetected for 60, your investigation is over before it begins.
• Forgetting identity provider logs. Sign-in and directory logs from your IdP are frequently where compromise first appears, and they often live outside the cloud provider console.
• No runbook. Knowing an alert fired is different from knowing what to do at 3am.

How to practice this hands on

Reading about log architecture is not the same as building one. Set up a free-tier or trial account and work through the full loop: enable audit logging, centralize it, deliberately perform a suspicious action such as creating an over-privileged role, then find that action in your logs and write a detection for it. That loop, repeated across AWS, Azure, and Google Cloud, is what separates candidates who can talk about detection from engineers who can build it.

This is exactly the kind of work built into the PrimeSec Academy curriculum, where logging, detection, and incident response run as hands-on labs across all three clouds rather than as slides. You can see the kind of build work involved on our projects page, and if you are unsure whether the program fits your background, the eligibility quiz takes a couple of minutes.

Frequently asked questions

What is the difference between logging and monitoring in cloud security?

Logging is the capture and storage of event records. Monitoring is the continuous analysis of those records to identify conditions that require attention. You can log without monitoring, which is common and dangerous, but you cannot meaningfully monitor without logging.

Which cloud logs should I enable first if I have a limited budget?

Start with the control plane audit log in every account, subscription, or project, because it is low volume, usually inexpensive, and covers the attack paths that matter most. Add identity provider sign-in logs next. Data plane logging comes third, enabled selectively on sensitive resources.

How long should I retain cloud security logs?

It depends on your regulatory obligations and your realistic detection time. A common approach is 30 to 90 days of hot searchable storage for investigation, plus one to seven years of cheaper archival storage for compliance. Check your specific framework requirements before choosing.

Do I need a SIEM, or is the cloud provider's native tooling enough?

Native tooling such as Microsoft Sentinel, AWS Security Hub with GuardDuty, or Google Security Command Center is sufficient for many single-cloud organizations. A third party SIEM becomes more attractive when you run multiple clouds, need to correlate with on-premises sources, or have specific compliance reporting requirements.

Can attackers delete cloud logs to cover their tracks?

They can try, which is why logs should be written to a separate account or project that the workload identity cannot delete from, protected with immutability controls, and monitored for any attempt to disable or modify logging configuration. Detection of logging being turned off is itself one of the highest value alerts you can build.

What skills do I need to work in cloud detection engineering?

You need a working knowledge of cloud provider audit log schemas, a query language such as KQL or SQL, familiarity with at least one SIEM, an understanding of common attacker techniques against cloud identities, and enough scripting ability to automate enrichment and response.

---

Logging is the foundation every other cloud security capability rests on. If you want to build that foundation through guided, hands-on labs across AWS, Azure, and Google Cloud, explore the PrimeSec Academy curriculum or enroll in the program.