GSM Propagation Models: Choosing a Model for Coverage Planning

Introduction to Cellular Propagation Modeling in GSM Networks

In GSM network engineering, radio frequency (RF) propagation models form the mathematical backbone for site selection, link budget formulation, and coverage map prediction. An empirical or statistical propagation model estimates signal attenuation as a electromagnetic wave travels from a Base Transceiver Station (BTS) antenna to a Mobile Station (MS). Choosing an inappropriate model can lead to costly operational mistakes: over-predicting coverage creates unexpected coverage holes and dropped calls, while under-predicting coverage results in unnecessary cell site builds and excessive capital expenditure.

While modern planners use computerized simulation platforms such as the GSM Coverage Planner and lightweight tools like the GSM Coverage Planner Web, the accuracy of any software output remains tied to the selected model and its underlying formulas. According to ITU-R Recommendation P.525, free-space path loss increases quadratically with both frequency and distance. However, actual terrestrial GSM environments introduce shadowing, diffraction, reflections, and atmospheric absorption that require empirical modifications to basic free-space equations.

This guide analyzes the most reliable GSM propagation models, outlines their mathematical limits and frequency applicability, and explains how to calibrate these models using field measurements collected via tools like GSM Drive Test.

Okumura-Hata Model Parameters and Applicable Limits

The Okumura-Hata model remains the foundational empirical model for macrocellular planning in the GSM 850 and GSM 900 bands. Derived by Yoshihisa Okumura from extensive field measurements in Tokyo and later formalized mathematically by Masaharu Hata, this model predicts median path loss for point-to-point terrestrial links in non-line-of-sight (NLOS) conditions.

Mathematical Validity Bounds

The standard Hata model is valid strictly within the following boundary conditions:

• Carrier Frequency ($f$): 150 MHz to 1,500 MHz (covering standard GSM 850 and GSM 900 allocations).
• Base Station Antenna Height ($h_b$): 30 meters to 200 meters.
• Mobile Station Antenna Height ($h_m$): 1 meter to 10 meters.
• Link Distance ($d$): 1 kilometer to 20 kilometers.

Core Formulas

For a medium-to-small city, basic path loss $L_{U}$ in decibels (dB) is expressed as:

L_U = 69.55 + 26.16 \log_{10}(f) - 13.82 \log_{10}(h_b) - a(h_m) + [44.9 - 6.55 \log_{10}(h_b)] \log_{10}(d)

Where $a(h_m)$ is the mobile station antenna height correction factor. For a medium-to-small city, $a(h_m)$ is calculated as:

a(h_m) = (1.1 \log_{10}(f) - 0.7) h_m - (1.56 \log_{10}(f) - 0.8)

For dense urban centers at frequencies $f \ge 300\text{ MHz}$, $a(h_m)$ takes a different form:

a(h_m) = 3.2 [\log_{10}(11.75 h_m)]^2 - 4.97

Suburban and Open Rural Environment Corrections

To adapt urban calculations for lower-density regions, subtract environmental correction factors from the urban path loss $L_{U}$:

• Suburban Area ($L_{SU}$): L_{SU} = L_U - 2 [\log_{10}(f / 28)]^2 - 5.4
• Open Rural Area ($L_{O}$): L_O = L_U - 4.78 [\log_{10}(f)]^2 + 18.33 \log_{10}(f) - 40.94

The primary advantage of the Okumura-Hata model is its low computational complexity and reliance on easily accessible macro parameters. However, its primary practical limitation is its total lack of terrain profile awareness and inability to model links under 1 km.

COST 231 Hata Model Extension for DCS 1800 Networks

When GSM expanded from 900 MHz into the 1800 MHz band (DCS 1800 in Europe and Asia) and 1900 MHz (PCS 1900 in the Americas), the classical Okumura-Hata model underestimated path loss because it was limited to 1,500 MHz. The European Cooperative for Scientific and Technical Research (COST 231 working group) extended Hata's equations to cover frequencies up to 2,000 MHz.

Parameter Ranges for COST 231 Hata

• Carrier Frequency ($f$): 1,500 MHz to 2,000 MHz.
• Base Station Antenna Height ($h_b$): 30 meters to 200 meters.
• Mobile Station Antenna Height ($h_m$): 1 meter to 10 meters.
• Link Distance ($d$): 1 kilometer to 20 kilometers.

COST 231 Formulation

Path loss $L_{C231}$ is expressed as:

L_{C231} = 46.3 + 33.9 \log_{10}(f) - 13.82 \log_{10}(h_b) - a(h_m) + [44.9 - 6.55 \log_{10}(h_b)] \log_{10}(d) + C_M

Where $C_M$ is a constant offset parameter defined as:

• 0 dB for medium-sized cities and suburban environments with moderate tree density.
• 3 dB for dense metropolitan centers and urban building concentrations.

The table below summarizes key differences between Okumura-Hata and COST 231 Hata when planning GSM cell radii:

ITU-R P.1546 Recommendation for Point-to-Area Predictions

Where Hata-based models rely on smooth terrain assumptions, ITU-R Recommendation P.1546 provides a statistical method for point-to-area field strength predictions for terrestrial services between 30 MHz and 3,000 MHz. It is maintained by ITU Study Group 3 and updated to account for mixed land-sea paths and irregular terrain profiles.

Key Features of ITU-R P.1546

Unlike simple formulas, ITU-R P.1546 relies on tabulated field strength curves normalized for 1 kW effective radiated power (ERP) as a function of distance, frequency, effective transmitter antenna height, and time percentages.

• Time and Location Variability: Supports prediction curves for 50%, 10%, and 1% of time availability, making it suitable for co-channel interference assessment as well as coverage design.
• Effective Antenna Height ($h_{eff}$): Evaluates transmitter height relative to average ground height between 3 km and 15 km along the specific radial path toward the receiver, rather than simple height above local ground.
• Terrain Clearance Angle (TCA): Applies corrections based on the elevation angle from the mobile receiver to the highest terrain obstruction within a 16 km radius.

For rural GSM planning across mixed terrain or coastal boundaries, ITU-R P.1546 offers superior accuracy over uncalibrated Okumura-Hata models, though it requires digital elevation models (DEM) to evaluate $h_{eff}$ and terrain clearance angles correctly.

Empirical vs. Deterministic Ray-Tracing Models in Practice

When selecting a propagation model, RF engineers must choose between fast empirical statistical models and complex deterministic models.

Empirical Models (Okumura-Hata, COST 231, ITU-R P.1546)

Empirical models compute loss using statistical fits to field measurements. They execute fast, making them ideal for large-scale rural or regional coverage predictions across thousands of square kilometers. However, they average out localized obstacle effects, producing inaccuracies in urban street canyons or complex terrain.

Deterministic and 3D Ray-Tracing Models

Deterministic models use 3D vector building vector maps and terrain profiles to trace individual wave paths, calculating specular reflections, knife-edge diffractions, and scattering effects (e.g., Fresnel zone clearance calculations). Models like Walfisch-Ikegami or 3D Shooting and Bouncing Rays (SBR) predict signal levels inside microcells and dense urban street corridors.

Field Rule of Thumb: Use COST 231 or ITU-R P.1546 for macrocell layout across suburban and rural environments. Reserve 3D ray-tracing models for high-density urban microcells where antenna heights are below rooftop levels ($h_b < 15\text{ m}$).

Calibrating Empirical Models with Field Measurement Data

An uncalibrated empirical propagation model typically exhibits a standard error ($\\sigma$) between 8 dB and 12 dB when compared against actual receiver signal levels (RxLev). Tuning model constants using real drive-test data reduces prediction error ($\\sigma < 6\text{ dB}$), aligning simulations with live network performance.

Standard Standard Log-Distance Calibration Equation

Most commercial planning tools represent tuned models using a modified log-distance equation:

L_{cal} = K_1 + K_2 \log_{10}(d) + K_3 \log_{10}(h_b) + K_4 \times \text{Diffraction} + K_5 \log_{10}(h_{eff}) + K_{clutter}

Where $K_1$ is the constant offset (dB) and $K_2$ is the path loss slope coefficient.

Calibration Procedure

1. Data Collection: Perform drive testing with a continuous log recorder (such as GSM Drive Test) operating on a dedicated continuous wave (CW) beacon or non-frequency-hopping GSM BCCH carrier. Ensure receiver sampling rate conforms to Lee's Sampling Theorem (40 to 50 samples over a $40\lambda$ distance interval) to average out fast Rayleigh fading.
2. Filtering: Remove invalid points, such as measurements taken when the vehicle was stationary, receiver saturation points near the site (< 100 meters), and samples below the receiver noise floor (-110 dBm).
3. Least-Squares Regression: Fit the measured RxLev values against logarithmic distance to calculate optimized coefficients $K_1$ and $K_2$.
4. Validation: Split field data into a 70% calibration dataset and a 30% verification dataset. Verify that the mean error ($\\mu$) approaches 0 dB and the standard deviation ($\\sigma$) drops below 6 dB.

Field Troubleshooting Sequence for Coverage Discrepancies

When field-measured RxLev significantly diverges from propagation predictions (exceeding a 10 dB error margin), follow this systematic field troubleshooting sequence:

Step 1: Verify Transmit Path Power and Attenuation

Measure actual BTS output power at the top of the cabinet using an RF power meter. Account for jumper losses, combiner losses, and main feeder attenuation. A loose connector or damaged feeder line can easily inject 3 dB to 10 dB of unmodeled signal loss before the antenna.

Step 2: Confirm Antenna Alignment and Electrical Tilt

Audit physical antenna azimuth and mechanical tilt at the tower top using the GSM Azimut Checker. Verify that remote electrical tilt (RET) settings in the base station controller match the simulation project file. An incorrect tilt offset of 2 degrees can alter main lobe footprint boundaries by hundreds of meters.

Step 3: Audit Terrain and Clutter Layer Resolution

Inspect clutter map alignment in the planning tool. Standard errors occur when low-resolution elevation maps (e.g., 90-meter SRTM) fail to register narrow ridges, deep valleys, or recent building construction blocking line-of-sight path propagation.

Step 4: Check Fresnel Zone Clearance

Analyze line-of-sight profiles for critical radio links. According to ITU-R Recommendation P.530, at least 60% clearance of the first Fresnel zone ($F_1$) is required to avoid diffraction attenuation over terrain obstacles:

R_1 = 17.32 \sqrt{\frac{d_1 d_2}{f \times D}}

Where $R_1$ is the first Fresnel zone radius in meters, $f$ is frequency in GHz, $d_1$ and $d_2$ are distances to the obstacle in km, and $D = d_1 + d_2$.

Step 5: Perform Mobile Rx Sensitivity and Calibration Checks

Verify that the drive-test receiver cable and antenna mounted on the test vehicle are undamaged. Recalibrate test terminal low-noise amplifiers (LNA) to rule out instrument measuring error.

Frequently asked questions