Colne is close to the southern entrance to the Aire Gap, the lowest crossing of the Pennine watershed. The M65 terminates west of the town and from here two main roads take traffic onwards towards the Yorkshire towns of Skipton (A56) and Keighley (A6068). Colne railway station is the terminus of the East Lancashire railway line.
Settlement in the area can be traced back to the Stone Age. A Mesolithic camp site, a Bronze Age burial site and stone tools from the Bronze and Stone Ages have been discovered at nearby Trawden. There are also the remains of an Iron Age fort, dating from the 6th century BC, above Colne at Castercliff.
Although a Roman road passes through nearby Barnoldswick, and some Roman coins have been discovered, there is no conclusive evidence of the Romans having occupied the area. There is, however, some debate among local historians as to whether the Romans may have stayed at Castercliff.
From the early 6th century to the late 10th century, Colne came under Northumbrian and then Viking rule, finally coming firmly under Norman control in the 11th century. Then, from the 1090s until 1311, the area was controlled by the de Lacys of Pontefract from their outpost at Clitheroe Castle. Pendle Forest and Trawden Forest date from this period; forests in those times being hunting grounds for royals and other nobles. St Bartholomew's Church dates from before 1122 when the town's market was located in the churchyard. The churchyard used to house the market cross and wooden stocks on wheels and people were placed in these on market days. The stocks are now located in the nearby library. The market cross is in Market Street.