Best practices for URL filtering service

This document describes best practices for using URL filtering service with Cloud Next Generation Firewall. Use these recommendations to create effective and maintainable firewall policies that help to prevent unintended filtering actions. This document assumes familiarity with URL filtering service concepts.

Best practices

The following table lists the best practices for using the URL filtering service:

Specify destination protocol and ports

Cloud NGFW applies the default allow or deny action in a URL filtering security profile to non-HTTP and non-HTTPS traffic that matches a firewall policy rule enforcing URL filtering. Specify the protocol TCP and the ports 80 and 443 in such a firewall policy rule.

Specifying the protocol and ports in the rule prevents a match with non-HTTP and non-HTTPS traffic, and therefore, prevents the default action from unintentionally allowing or denying this traffic.

Place URL filtering rules at lowest priorities

To ensure that firewall policy rules that don't use Layer 7 inspection are evaluated first, place rules that enforce URL filtering security profiles at the lowest priorities in a firewall policy. This approach prevents the default action in a URL filtering security profile from accidentally allowing or denying traffic.

For example, to allow any virtual machine (VM) instance to communicate with www.example.com , create a URL filtering security profile that allows this URL. And then, add a rule with a low priority, such as 500 that applies a security profile group with this security profile.

Priority	Direction	Protocol	Target	Source	Destination	Destination port	Action
150	Egress	TCP	All	10.0.0.0/8	10.0.0.0/8	80, 443	Allow
200	Ingress	TCP	All	Any	10.0.0.0/8	80, 443	Allow
300	Egress	TCP	All	Any	199.36.153.8/30	80, 443	Allow
500	Egress	TCP	All	Any	Any	80, 443	Apply a security profile group with URL filtering security profile that allows the URL www.example.com

Use network contexts to separate traffic

The default action of a security profile can allow or deny traffic that doesn't match the defined URL filters. To prevent unintended default actions, use network contexts to create separate URL filtering firewall policy rules for east-west (non-internet-bound) traffic and internet-bound traffic. Network contexts make the rule match criteria more granular, and therefore, prevent the default action from accidentally allowing or denying traffic.

In the following example, the non-internet network context prevents the rule with the priority of 500 from matching and accidentally allowing or denying internet-bound traffic, which the rule with a priority of 600 manages separately.

Priority	Direction	Network context	Protocol	Target	Source	Destination	Destination port	Action
500	Egress	Non-internet	TCP	All	Any	Any	80, 443	Apply a security profile group with URL filtering security profile 1
600	Egress	Internet	TCP	All	Any	Any	80, 443	Apply a security profile group with URL filtering security profile 2

Use FQDN objects

Use destination FQDN objects along with URL filtering security profiles to avoid accidental traffic matches for the firewall policy rules, and therefore, prevent the security profile's default action from allowing or denying any unintended traffic.

Use FQDNs to add pinhole exceptions

A pinhole exception in a firewall allows a specific type of traffic to pass through an otherwise closed security perimeter. To create such an exception that allows a specific source to access a website, use a destination FQDN object and a URL filtering security profile in your firewall policy rule.

A pinhole exception rule matches traffic from the source to the FQDN IP before applying URL filtering. Other traffic from the source doesn't match, preventing unrelated traffic from the security profile's default action.

In the following example, the rule matches traffic from the specified IP range to the IP address of example.com , then applies URL filtering to the matched traffic.

Priority	Direction	Network context	Protocol	Target	Source	Destination	Destination port	Action
100	Egress	Internet	TCP	All	192.168.1.0/24	FQDN = example.com	80, 443	Apply a security profile group with URL filtering security profile that allows the URL www.example.com

Use FQDNs for selective traffic inspection

To selectively inspect traffic based on the destination domains, use FQDN objects instead of URL filtering security profiles.

URL filtering security profiles don't support selective traffic inspection. If you use URL filtering security profiles for traffic selection, the default action of the profile might unintentionally allow or deny traffic.

For example, to inspect traffic to example.com for threats without affecting traffic to examplepetstore.com , create separate firewall policy rules with priority 100 and 200 . For each rule, use a destination FQDN object and apply the threat prevention security profile only to the rule with the example.com destination. This approach prevents traffic to examplepetstore.com from matching the rule with the priority of 100 .

The following table shows the incorrect configuration:

Priority	Direction	Protocol	Target	Source	Destination	Destination port	Action
100	Egress	TCP	All	Any	Any	80, 443	Apply a security profile group with URL filtering security profile 1 that allows the URL example.com and a threat prevention security profile that inspects traffic to the URL example.com
200	Egress	TCP	All	Any	Any	80, 443	Apply a security profile group with URL filtering security profile 2 that allows the URL examplepetstore.com

The following table shows the correct configuration:

Priority	Direction	Protocol	Target	Source	Destination	Destination port	Action
100	Egress	TCP	All	Any	FQDN = example.com	80, 443	Apply a security profile group with threat prevention security profile that inspects traffic to the URL example.com
200	Egress	TCP	All	Any	FQDN = examplepetstore.com	80, 443	Apply a security profile group with URL filtering security profile that allows the URL examplepetstore.com

Avoid creating rules based on a single resource

To address temporary requirements, avoid creating firewall policy rules with URL filtering that use a single resource as a source.

If you keep these rules, the default action of a URL filtering security profile might allow or deny unintended traffic. If you don't delete them immediately, these rules can also lead to policy sprawl over time.

In the following example, a rule with a higher priority of 100 matches and allows traffic from 192.168.1.1 to examplepetstore.com . This evaluation occurs before the traffic reaches rule with a priority of 101 , which is configured to block that traffic.

Priority	Direction	Network context	Protocol	Target	Source	Destination	Destination port	Action
100	Egress	Internet	TCP	All	192.168.1.1/32	Any	80, 443	Apply a security profile group with URL filtering security profile 1 that blocks the URL example.com
101	Egress	Non-internet	TCP	All	192.168.1.0/24	Any	80, 443	Apply a security profile group with URL filtering security profile 2 that blocks the URL examplepetstore.com and allows the URL example.org

However, if you need to retain such a rule, use a destination FQDN object. For example, to avoid matching traffic from 192.168.1.1 to examplepetstore.com , add the FQDN object example.com to the rule with the priority of 100 .

Priority	Direction	Network context	Protocol	Target	Source	Destination	Destination port	Action
100	Egress	Internet	TCP	All	192.168.1.1/32	FQDN = example.com	80, 443	Apply a security profile group with URL filtering security profile 1 that blocks the URL example.com
101	Egress	Non-internet	TCP	All	192.168.1.0/24	Any	80, 443	Apply a security profile group with URL filtering security profile 2 that blocks the URL examplepetstore.com and allows the URL example.org

Use FQDN-based filtering for non-HTTP protocols

Use FQDN-based filtering for non-HTTP and non-HTTPS protocols, such as RDP or SSH. URL filtering only works with HTTP or HTTPS traffic.

Avoid using URL filtering in hierarchical firewall policies

To prevent unintended traffic matches, avoid using URL filtering security profiles in the hierarchical firewall policies . Instead, use URL filtering security profiles in the global or regional network firewall policies.

Using a URL filtering security profile in a hierarchical firewall policy rule can cause traffic for a broad set of targets to match that rule. This broad matching can cause the security profile's default action to unintentionally allow or deny traffic.

Create separate security profiles for scale

To stay within the supported limit for the number of domains or matcher strings in a URL filtering security profile, create separate security profiles that group domains based on a specific criteria. For example, group domains based on the traffic type or the location and nature of the destination domain, such as east-west (non-internet-bound) and internet-bound traffic.

This segmentation helps ensure optimal performance and manageability when you use many domains.

For example, create one profile for internal or Google Cloud domains and another for external domains on the public internet. A separate profile for internal domains keeps internal traffic routing and security policies distinct. A separate profile for external domains lets you apply appropriate security measures, such as threat intelligence feeds or stricter egress controls, to external traffic.

What's next

• URL filtering service
• Create and manage URL filtering security profiles
• Create and manage threat prevention security profiles
• Create and manage security profile groups
• Use global network firewall policies and rules
• Use regional network firewall policies and rules