Skip to content

Commit 34c77f7

Browse files
committed
add source ashford_gov_uk
1 parent ddc06ca commit 34c77f7

File tree

4 files changed

+172
-1
lines changed

4 files changed

+172
-1
lines changed

README.md

+1
Original file line numberDiff line numberDiff line change
@@ -704,6 +704,7 @@ Waste collection schedules in the following formats and countries are supported.
704704
- [Aberdeenshire Council](/doc/source/aberdeenshire_gov_uk.md) / aberdeenshire.gov.uk
705705
- [Amber Valley Borough Council](/doc/source/ambervalley_gov_uk.md) / ambervalley.gov.uk
706706
- [Ashfield District Council](/doc/source/ashfield_gov_uk.md) / ashfield.gov.uk
707+
- [Ashford Borough Council](/doc/source/ashford_gov_uk.md) / ashford.gov.uk
707708
- [Basildon Council](/doc/source/basildon_gov_uk.md) / basildon.gov.uk
708709
- [Basingstoke and Deane Borough Council](/doc/source/basingstoke_gov_uk.md) / basingstoke.gov.uk
709710
- [Bath & North East Somerset Council](/doc/source/bathnes_gov_uk.md) / bathnes.gov.uk
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
import requests
2+
from waste_collection_schedule import Collection # type: ignore[attr-defined]
3+
from bs4 import BeautifulSoup, Tag
4+
import re
5+
from datetime import datetime
6+
7+
TITLE = "Ashford Borough Council"
8+
DESCRIPTION = "Source for Ashford Borough Council."
9+
URL = "https://ashford.gov.uk"
10+
TEST_CASES = {
11+
"100060796052": {
12+
"uprn": 100060796052,
13+
"postcode": "TN23 3DY"
14+
},
15+
"100060780440": {
16+
"uprn": "100060780440",
17+
"postcode": "TN24 9JD"
18+
},
19+
"100062558476": {
20+
"uprn": "100062558476",
21+
"postcode": "TN233LX"
22+
},
23+
}
24+
25+
26+
ICON_MAP = {
27+
"household refuse": "mdi:trash-can",
28+
"food waste": "mdi:food",
29+
"garden waste": "mdi:leaf",
30+
"recycling": "mdi:recycle",
31+
}
32+
33+
34+
API_URL = "https://secure.ashford.gov.uk/wastecollections/collectiondaylookup/"
35+
36+
37+
class Source:
38+
def __init__(self, postcode: str, uprn: str | int):
39+
self._uprn = str(uprn).strip()
40+
self._postcode = str(postcode).strip()
41+
42+
self._post_code_args = {"CollectionDayLookup2$Button_PostCodeSearch": "Continue+>"}
43+
self._uprn_args = {"CollectionDayLookup2$Button_SelectAddress": "Continue+>"}
44+
self._post_code_args["CollectionDayLookup2$TextBox_PostCode"] = self._postcode
45+
self._uprn_args["CollectionDayLookup2$DropDownList_Addresses"] = self._uprn
46+
47+
def fetch(self):
48+
# get json file
49+
s = requests.Session()
50+
r = s.get(API_URL)
51+
r.raise_for_status()
52+
53+
soup: BeautifulSoup = BeautifulSoup(r.text, "html.parser")
54+
55+
viewstate = soup.find("input", id="__VIEWSTATE")
56+
viewstate_generator = soup.find("input", id="__VIEWSTATEGENERATOR")
57+
58+
if not viewstate or type(viewstate) != Tag or not viewstate_generator or type(viewstate_generator) != Tag:
59+
raise Exception("could not get valid data from ashford.gov.uk")
60+
61+
self._post_code_args["__VIEWSTATE"] = str(viewstate["value"])
62+
63+
self._post_code_args["__VIEWSTATEGENERATOR"] = str(viewstate_generator["value"])
64+
self._uprn_args["__VIEWSTATEGENERATOR"] = str(viewstate_generator["value"])
65+
66+
r = s.post(API_URL, data=self._post_code_args)
67+
r.raise_for_status()
68+
69+
soup: BeautifulSoup = BeautifulSoup(r.text, "html.parser")
70+
viewstate = soup.find("input", id="__VIEWSTATE")
71+
if not viewstate or type(viewstate) != Tag:
72+
raise Exception("could not get valid data from ashford.gov.uk")
73+
74+
self._uprn_args["__VIEWSTATE"] = str(viewstate["value"])
75+
76+
r = s.post(API_URL, data=self._uprn_args)
77+
if r.status_code != 200:
78+
raise Exception(
79+
f"could not get correct data for your postcode ({self._postcode}). check {API_URL} to validate your arguments.")
80+
81+
soup: BeautifulSoup = BeautifulSoup(r.text, "html.parser")
82+
83+
bin_tables = soup.find_all("table")
84+
if bin_tables == []:
85+
raise Exception(
86+
f"could not get valid data from ashford.gov.uk. is your UPRN ({self._uprn}) correct for postcode ({self._postcode})? check https://uprn.uk/{self._uprn} and {API_URL}")
87+
88+
entries = []
89+
for bin_table in bin_tables:
90+
bin_text = bin_table.find(
91+
"td", id=re.compile("CollectionDayLookup2_td_"))
92+
if not bin_text:
93+
continue
94+
95+
bin_type_soup = bin_text.find("b")
96+
97+
if not bin_type_soup:
98+
continue
99+
bin_type: str = bin_type_soup.text.strip()
100+
101+
date_soup = bin_text.find("span", id=re.compile(r"CollectionDayLookup2_Label_\w*_Date"))
102+
if not date_soup or not " " in date_soup.text.strip():
103+
continue
104+
date_str: str = date_soup.text.strip()
105+
try:
106+
date = datetime.strptime(
107+
date_str.split(" ")[1], "%d/%m/%y").date()
108+
109+
except ValueError:
110+
continue
111+
112+
icon = ICON_MAP.get(bin_type.split("(")[0].strip().lower())
113+
entries.append(Collection(date=date, t=bin_type, icon=icon))
114+
115+
return entries

doc/source/ashford_gov_uk.md

+55
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# Ashford Borough Council
2+
3+
Support for schedules provided by [Ashford Borough Council](https://ashford.gov.uk), serving Ashford, UK.
4+
5+
## Configuration via configuration.yaml
6+
7+
```yaml
8+
waste_collection_schedule:
9+
sources:
10+
- name: ashford_gov_uk
11+
args:
12+
uprn: UPRN
13+
postcode: POSTCODE
14+
15+
```
16+
17+
### Configuration Variables
18+
19+
**uprn**
20+
*(String | Integer) (required)*
21+
22+
**postcode**
23+
*(String) (required)*
24+
25+
## Example
26+
27+
```yaml
28+
waste_collection_schedule:
29+
sources:
30+
- name: ashford_gov_uk
31+
args:
32+
uprn: 100060796052
33+
postcode: TN23 3DY
34+
35+
```
36+
37+
## How to get the source argument
38+
39+
Use your postcode as `postcode` argument.
40+
41+
### How to get your UPRN
42+
43+
#### From external website
44+
45+
Find the parameter of your address using <https://secure.ashford.gov.uk/wastecollections/collectiondaylookup/> and write them exactly like on the web page.
46+
47+
#### From browser request analasys
48+
49+
- Go to <https://secure.ashford.gov.uk/wastecollections/collectiondaylookup/>.
50+
- Insert you postcode and click `Continue`.
51+
- Open your browsers Inspection tools (`F12` or `right click -> inspect`) and select the network tab.
52+
- Select your address and click `continue`.
53+
- You should now see multiple requests in the network tab. The first one should be (`POST`) `collectiondaylookup/`, select it.
54+
- Select the `Payload` tab of this request.
55+
- You can find your UPRN after `CollectionDayLookup2$DropDownList_Addresses:`

info.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ Waste collection schedules from service provider web sites are updated daily, de
3030
| Slovenia | Moji odpadki, Ljubljana |
3131
| Sweden | Affärsverken, Jönköping - June Avfall & Miljö, Landskrona - Svalövs Renhållning, Lerum Vatten och Avlopp, Linköping - Tekniska Verken, Region Gotland, Ronneby Miljöteknik, Samverkan Återvinning Miljö (SÅM), SRV Återvinning, SSAM, Sysav Sophämntning, Uppsala Vatten och Avfall AB, VA Syd Sophämntning |
3232
| Switzerland | A-Region, Andwil, Appenzell, Berg, Bühler, Eggersriet, Gais, Gaiserwald, Goldach, Grosswangen, Grub, Heiden, Herisau, Horn, Hundwil, Häggenschwil, Lindau, Lutzenberg, Muolen, Mörschwil, Münchenstein, Real Luzern, Rehetobel, Rorschach, Rorschacherberg, Schwellbrunn, Schönengrund, Speicher, Stein, Steinach, Teufen, Thal, Trogen, Tübach, Untereggen, Urnäsch, Wald, Waldkirch, Waldstatt, Wittenbach, Wolfhalden |
33-
| United Kingdom | Aberdeenshire Council, Amber Valley Borough Council, Ashfield District Council, Basildon Council, Basingstoke and Deane Borough Council, Bath & North East Somerset Council, Bedford Borough Council, Binzone, Blackburn with Darwen Borough Council, Bracknell Forest Council, Bradford Metropolitan District Council, Braintree District Council, Breckland Council, Bristol City Council, Broadland District Council, Buckinghamshire Waste Collection - Former Chiltern, South Bucks or Wycombe areas, Cambridge City Council, Canterbury City Council, Central Bedfordshire Council, Cherwell District Council, Cheshire East Council, Chesterfield Borough Council, Chichester District Council, City of Doncaster Council, City of York Council, Colchester City Council, Cornwall Council, Croydon Council, Derby City Council, East Cambridgeshire District Council, East Herts Council, East Northamptonshire and Wellingborough, East Riding of Yorkshire Council, Eastbourne Borough Council, Elmbridge Borough Council, Environment First, Exeter City Council, Fareham Council, FCC Environment, Fenland District Council, Fife Council, Gateshead Council, Glasgow City Council, Guildford Borough Council, Harborough District Council, Harlow Council, Herefordshire City Council, Horsham District Council, Huntingdonshire District Council, Kirklees Council, Leicester City Council, Lewes District Council, Lisburn and Castlereagh City Council, Liverpool City Council, London Borough of Bexley, London Borough of Bromley, London Borough of Lewisham, London Borough of Merton, Maidstone Borough Council, Maldon District Council, Manchester City Council, Mid-Sussex District Council, Middlesbrough Council, Newcastle City Council, Newcastle Under Lyme Borough Council, Newport City Council, North Herts Council, North Kesteven District Council, North Lincolnshire Council, North Somerset Council, Nottingham City Council, Oxford City Council, Peterborough City Council, Portsmouth City Council, Reading Council, Redbridge Council, Reigate & Banstead Borough Council, Richmondshire District Council, Rotherham Metropolitan Borough Council, Runnymede Borough Council, Rushcliffe Brough Council, Rushmoor Borough Council, Salford City Council, Sheffield City Council, South Cambridgeshire District Council, South Derbyshire District Council, South Gloucestershire Council, South Hams District Council, South Norfolk Council, South Oxfordshire District Council, South Tyneside Council, Southampton City Council, Stevenage Borough Council, Stockport Council, Stockton-on-Tees Borough Council, Swindon Borough Council, Telford and Wrekin Council, Tewkesbury Borough Council, The Royal Borough of Kingston Council, Tonbridge and Malling Borough Council, Uttlesford District Council, Vale of White Horse District Council, Walsall Council, Waverley Borough Council, Wealden District Council, Welwyn Hatfield Borough Council, West Berkshire Council, West Devon Borough Council, West Dunbartonshire Council, Wigan Council, Wiltshire Council, Wirral Council, Wyre Forest District Council |
33+
| United Kingdom | Aberdeenshire Council, Amber Valley Borough Council, Ashfield District Council, Ashford Borough Council, Basildon Council, Basingstoke and Deane Borough Council, Bath & North East Somerset Council, Bedford Borough Council, Binzone, Blackburn with Darwen Borough Council, Bracknell Forest Council, Bradford Metropolitan District Council, Braintree District Council, Breckland Council, Bristol City Council, Broadland District Council, Buckinghamshire Waste Collection - Former Chiltern, South Bucks or Wycombe areas, Cambridge City Council, Canterbury City Council, Central Bedfordshire Council, Cherwell District Council, Cheshire East Council, Chesterfield Borough Council, Chichester District Council, City of Doncaster Council, City of York Council, Colchester City Council, Cornwall Council, Croydon Council, Derby City Council, East Cambridgeshire District Council, East Herts Council, East Northamptonshire and Wellingborough, East Riding of Yorkshire Council, Eastbourne Borough Council, Elmbridge Borough Council, Environment First, Exeter City Council, Fareham Council, FCC Environment, Fenland District Council, Fife Council, Gateshead Council, Glasgow City Council, Guildford Borough Council, Harborough District Council, Harlow Council, Herefordshire City Council, Horsham District Council, Huntingdonshire District Council, Kirklees Council, Leicester City Council, Lewes District Council, Lisburn and Castlereagh City Council, Liverpool City Council, London Borough of Bexley, London Borough of Bromley, London Borough of Lewisham, London Borough of Merton, Maidstone Borough Council, Maldon District Council, Manchester City Council, Mid-Sussex District Council, Middlesbrough Council, Newcastle City Council, Newcastle Under Lyme Borough Council, Newport City Council, North Herts Council, North Kesteven District Council, North Lincolnshire Council, North Somerset Council, Nottingham City Council, Oxford City Council, Peterborough City Council, Portsmouth City Council, Reading Council, Redbridge Council, Reigate & Banstead Borough Council, Richmondshire District Council, Rotherham Metropolitan Borough Council, Runnymede Borough Council, Rushcliffe Brough Council, Rushmoor Borough Council, Salford City Council, Sheffield City Council, South Cambridgeshire District Council, South Derbyshire District Council, South Gloucestershire Council, South Hams District Council, South Norfolk Council, South Oxfordshire District Council, South Tyneside Council, Southampton City Council, Stevenage Borough Council, Stockport Council, Stockton-on-Tees Borough Council, Swindon Borough Council, Telford and Wrekin Council, Tewkesbury Borough Council, The Royal Borough of Kingston Council, Tonbridge and Malling Borough Council, Uttlesford District Council, Vale of White Horse District Council, Walsall Council, Waverley Borough Council, Wealden District Council, Welwyn Hatfield Borough Council, West Berkshire Council, West Devon Borough Council, West Dunbartonshire Council, Wigan Council, Wiltshire Council, Wirral Council, Wyre Forest District Council |
3434
| United States of America | Albuquerque, New Mexico, USA, City of Oklahoma City, City of Pittsburgh, Louisville, Kentucky, USA, Newark, Delaware, USA, Olympia, Washington, USA, ReCollect, Recycle Coach, Republic Services, Seattle Public Utilities, Tucson, Arizona, USA |
3535
<!--End of country section-->
3636

0 commit comments

Comments
 (0)