The basic Python distribution is not shipped with the geo-processing modules bundled with GDAL. As such, after installation of Python and the GDAL, the relevant geo-processing Python bindings needs to be installed.
One advanced feature of the GDAL Python bindings not found in the other language bindings (such as C#, Perl) is integration with the Python numerical array facilities. For instance, the gdal.Dataset.ReadAsArray() method can be used to read raster data as numerical arrays, ready to use with the Python numerical array capabilities. More information about these bindings can be found here.
GDAL bindings examples
The gdal_merge.py script is an example of GDAL Python bindings, which takes the paths of the GeoTIFFs you want to combine as inputs and creates a single GeoTIFF as output. If you do not specify a filename with the -o option, the script will save the result as out.tif:
gdal_merge.py image1.tif image2.tif -o merged.tif
Another example is gdal_polygonize.py by which you can extract polygons from a raster:
gdal_polygonize.py input.tif -f "GeoJSON" output.json
For more example, visit the gdal-cheat-sheet.