v1.6.1; bug fixes, new api endpoint, test case fixes, get_script changes

amckenna41 · Jun 7, 2024 · 08f27f6 · 08f27f6
1 parent ae02aba
commit 08f27f6
Show file tree

Hide file tree

Showing 16 changed files with 927 additions and 720 deletions.
diff --git a/.github/workflows/CHANGELOG.md b/.github/workflows/CHANGELOG.md
@@ -1,9 +1,25 @@
 # Change Log
 
-### 1.7.0 - TBC
+## 1.7.0 - TBC
 
 ### Added
 - Add list of cities for each subdivision using the country-states-city API
+- Added __version__ attribute to ISO3166_2() class
+
+## v1.6.1 - June 2024
+
+### Added
+- list_subdivisions endpoint added to API that returns list of all subdivision codes per country
+- Unit tests for flag_url function in update_subdivisions script
+- Separate function for extracting and parsing data attributes from RestCountries API
+- Added raise_for_status error catcher for requests library
+
+### Changed
+- Rotate user agent headers for any scripts using requests.get 
+
+### Fixed
+- Error in request URL for RestCountries API in update_subdivisions script
+- Raise TypeError if invalid data type input to export_iso3166_2 function rather than system crashing
 
 
 ## v1.6.0 - June 2024
@@ -45,7 +61,8 @@
 - If a custom subdivision object is input via its respective function, latLng attribute is set to 3d.p 
 - Fixed syntax of some function parameters that can take multiple data types
 - Fixed parameter typing syntax for some function parameters that can be multiple data types
-
+- Error when adding a new subdivision, now will raise an error if the input parent code is invalid/not a country subdivision
+- In get_flag_url function in update_subdivision script, you can pass in the full subdivision code or just the RHS of it
 
 ## v1.5.4 - March 2024
 
@@ -72,7 +89,6 @@
 
 ## v1.4.0 - December 2023
 
-
 ### Added
 - readthedocs documentation added
 - Search for a particular subdivision via its subdivision name, finding the closest match using difflib libary
@@ -81,7 +97,6 @@
 
 
 ### Changed
-- 
 - Software package description updated
 - Changed software license to MIT
 - Remove subdivision_parent_codes function that returned this list of parent codes per subdivision

diff --git a/.github/workflows/build_test.yml b/.github/workflows/build_test.yml
@@ -149,6 +149,7 @@ jobs:
       - name: Install dependencies
         run: |
           python -m pip install --upgrade pip
+          if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
           pip3 install pytest pytest-cov codecov
 
       #generate codecov report using pytest

diff --git a/API.md b/API.md
@@ -14,7 +14,7 @@ The other endpoints available in the API are:
 * https://iso3166-2-api.vercel.app/api/subdivision/<input_subdivision>
 * https://iso3166-2-api.vercel.app/api/name/<input_subdivision_name>
 
-Five paths/endpoints are available in the API - `/api/all`, `/api/alpha`, `/api/country_name`, `/api/subdivision` and `/api/name`.
+Six paths/endpoints are available in the API - `/api/all`, `/api/alpha`, `/api/country_name`, `/api/subdivision`, `/api/name` and `/api/list_subdivisions`.
 
 * `/api/all`: get all of the ISO 3166 subdivision data for all countries.
 
@@ -26,6 +26,8 @@ Five paths/endpoints are available in the API - `/api/all`, `/api/alpha`, `/api/
 
 * `/api/name/`: get all of the ISO 3166 subdivision data for 1 or more ISO 3166-2 subdivision names, e.g `/api/name/Derry`. You can also input a comma separated list of subdivision name from the same or different countries and the data for each will be returned e.g `/api/name/Paris,Frankfurt,Rimini`. A closeness function is utilised to find the matching subdivision name, if no exact name match found then the most approximate subdivisions will be returned. Some subdivisions may have the same name, in this case each subdivision and its data will be returned e.g `/api/name/Saint George` (this example returns 5 subdivisions). This endpoint also has the likeness score (`?likeness=`) query string parameter that can be appended to the URL. This can be set between 1 - 100, representing a % of likeness to the input name the return subdivisions should be, e.g: a likeness score of 90 will return fewer potential matches whose name only match to a high degree compared to a score of 10 which will create a larger search space, thus returning more potential subdivision matches. A default likeness of 100 (exact match) is used, if no matching subdivision is found then this is reduced to 90. If an invalid subdivision name that doesn't match any is input then an error will be raised.
 
+* `/api/list_subdivisions`: get list of all the subdivision codes for all countries. 
+
 * `/api`: main homepage and API documentation.
 
 A demo of the software and API is available [here][demo].
@@ -432,6 +434,45 @@ function getData() {
 var data = JSON.parse(this.response)
 ```
 
+Get all ISO 3166-2 subdivision codes for all countries 
+------------------------------------------------------
+
+### Request
+`GET /api/list_subdivisions/Tikistan`
+
+    curl -i https://iso3166-2-api.vercel.app/api/list_subdivisions
+
+### Response
+    HTTP/2 200 
+    content-type: application/json
+    date: Sat, 03 Feb 2024 15:03:11 GMT
+    server: Vercel
+    content-length: 
+
+    {"AD":{"AD-02","AD-03"...}}
+
+### Python
+```python
+import requests
+
+request_url = "https://iso3166-2-api.vercel.app/api/list_subdivisions/"
+
+all_request = requests.get(request_url)
+all_request.json() 
+```
+
+### Javascript
+```javascript
+function getData() {
+  const response = 
+    await fetch(`https://iso3166-updates.com/api/list_subdivisions`); 
+  const data = await response.json()
+}
+
+// Begin accessing JSON data here
+var data = JSON.parse(this.response)
+```
+
 [Back to top](#TOP)
 
 [demo]: https://colab.research.google.com/drive/1btfEx23bgWdkUPiwdwlDqKkmUp1S-_7U?usp=sharing
diff --git a/README.md b/README.md
@@ -8,7 +8,7 @@
 [![Documentation Status](https://readthedocs.org/projects/iso3166-2/badge/?version=latest)](https://iso3166-2.readthedocs.io/en/latest/?badge=latest)
 [![License: MIT](https://img.shields.io/github/license/amckenna41/iso3166-2)](https://opensource.org/licenses/MIT)
 [![Issues](https://img.shields.io/github/issues/amckenna41/iso3166-2)](https://github.com/amckenna41/iso3166-2/issues)
-[![Size](https://img.shields.io/github/repo-size/amckenna41/iso3166-2)](https://github.com/amckenna41/iso3166-2)
+<!-- [![Size](https://img.shields.io/github/repo-size/amckenna41/iso3166-2)](https://github.com/amckenna41/iso3166-2) -->
 <!-- [![Commits](https://img.shields.io/github/commit-activity/w/amckenna41/iso3166-2)](https://github.com/iso3166-2) -->
 <!-- [![codecov](https://codecov.io/gh/amckenna41/pySAR/branch/master/graph/badge.svg?token=4PQDVGKGYN)](https://codecov.io/gh/amckenna41/pySAR) -->
 

diff --git a/ToDo.md b/ToDo.md
@@ -21,7 +21,6 @@
 - [X] Reorder software metadata in setup.py to be in order of main func, create __description__ var.
 - [X] Add spcae in between ISO3166
 - [X] Reorder subdivision codes in json output - alphabetically and or numerically.
-- [ ] 
 - [X] In unit tests, reorder subdivision names/codes arrays into alphabetcal/numerical order.
 - [X] In subdivisions in json, add link to flag in iso3166-flag-icons, if no flag then null, skip if status code != 200.
 - [X] Check for updates by calling export function and comparing new and old jsons.
@@ -278,7 +277,7 @@ https://iso3166-updates.com/api/year/2020 : up-to-date.
 - [X] Double check not doing restcall for each subdivision and just once per country.
 - [X] In update_subdivisions, swap out row['country-code] for country_code var.
 - [X] In get_script, add more tests for subdivisions that are added/amended/deleted 
-- [ ] Read over update_subdiv script.
+- [X] Read over update_subdiv script.
 - [X] For subdivisions whose subdivision codes have been updated in the update_subdivisions script, need to redo the script to get the flag URL e.g. GT-01, GT-02 ... KZ-11. Will need to rerun script to incorporate changes.
 - [X] Put flag icons get request code into its own function, used 3 times in get scripts.
 - [X] Add if __name__ == "main" for update_subdivisions, so it can be directly called from cmd.
@@ -293,20 +292,19 @@ https://iso3166-updates.com/api/year/2020 : up-to-date.
 - [X] Move info about iso3166_2_scripts from main readme.
 - [X] For scripts dir, add usage examples including terminal and importing via module import.
 - [X] In update_subdivisions, raise error if no subdivision code input.
-- [ ] Error when adding new subdivision data via cmd line/terminal - some attributes not being added to output object, error when importing parent_code parameter as None.
+- [X] Error when adding new subdivision data via cmd line/terminal - some attributes not being added to output object, error when importing parent_code parameter as None:
 - [X] When adding new subdivision data via cmd line/terminal, latLng attribute may be a string of a list, change to array (copy jsom is original)
-- [ ] When adding new subdivision data via cmd line/terminal, need to reorder attributes to normal order.
+- [X] When adding new subdivision data via cmd line/terminal, need to reorder attributes to normal order.
 - [X] Add separate unit test instance for update_subdivisions script.
 - [X] Run bandit and package security check
 - [X] Raise error in update_subdivisons function when comma separated list of alpha codes input, only one should be input? Add unit tests
 - [X] In update_subdivisions, when making custom change to object, double check that custom latLng values are set to 3.dp.
 - [X] In cusom_subdivisions, when making custom change to object, double check that custom latLng values are set to 3.dp.
-- [ ] In update_subdivisons, when adding new subdivison, validate parentCode, if it doesnt exist raise error, unless current code being added is same as parent code input. 
 - [X] Add CHANGELOG.md - https://keepachangelog.com/en/1.0.0/
-- [ ] In test_iso3166_2 when testing custom_subdivision function, don't directly edit existing iso3166-2.json, create a hard copy and pass in the duplicate file name to function.
+- [X] In test_iso3166_2 when testing custom_subdivision function, don't directly edit existing iso3166-2.json, create a hard copy and pass in the duplicate file name to function.
 - [X] In test_iso3166_2 test format of each subdivision code XX-Y, XX-YY, XX-YYY etc.
 - [X] Change flagUrl attribute to just flag?
-- [ ] In custom_subdivisons function, add a flag/parameter whereby if set it creates a hard copy of the existing iso3166-2.json object so it is not directly overwritten. Issue when using the software and iso3166-2.json will not contain added custom subdivision.
+- [X] In custom_subdivisons function, add a flag/parameter whereby if set it creates a hard copy of the existing iso3166-2.json object so it is not directly overwritten. Issue when using the software and iso3166-2.json will not contain added custom subdivision.
 - [ ] In test_get script, create a request session in setup and close session in tearDown to stop resource warning SSL socket no closed - https://stackoverflow.com/questions/48160728/resourcewarning-unclosed-socket-in-python-3-unit-test
 - [X] In iso3166_2.py add repr class function, add more info to __str__. Add unit tests
 - [X] In custom_subdivisions function, allow for the amendment of an exsting subdivision code, e.g changing its local name, flagUrl, latLng etc. Mention on readme that if erroneous data is found for an attribute/subdivision, you can raise an Issue but you could also implement it immediately into the object via the custom_subdivisions function.
@@ -333,10 +331,22 @@ https://iso3166-updates.com/api/year/2020 : up-to-date.
 - [X] In custom_subdivision function, alpha_code and subdivision_code requried
 - [X] In update_subdivisons, alpha_code and subdivison not required because of subdivisons_csv param
 - [X] If custom path input to ISO3166_2(), add validation to see if path already has the full path to object in it, else append filepath to it.
-- [ ] when packaging, download from testpypi, checking correct files/dirs are downloaded, before publishing to pypi.
-- [ ] When adding restCountries keys to output, ensure they are in alphabetical order and appended to the end of the columns.
+- [X] when packaging, download from testpypi, checking correct files/dirs are downloaded, before publishing to pypi.
+- [X] When adding restCountries keys to output, ensure they are in alphabetical order and appended to the end of the columns.
 - [X] Rerun script with new flag atttribute, update filesize and execution time on readme.
 - [X] Read over docs
+- [X] Remove multiple return data types.
+- [X] Update supported python version >3.10
+- [X] Update 1.6.0 - 1.6.1
+- [X] In update_subdivisons, when adding new subdivison, validate parentCode, if it doesnt exist raise error, unless current code being added is same as parent code input. 
+- [X] When updating_subdivision, can add restcountries keys even if other objects don't have these keys
+- [X] Test for get_flag_url function
+- [X] In get_flag URL, if only RHS of subdivision code input to its parameter, prepend alpha code to it.
+- [X] Issue with nested objects in RestCountries - new function that parses the desired attributes
+- [X] In restcountries request, raise for status to catch 404 error
+- [X] Test adding restcountries keys when amending a subdivision code.
+- [X] For some nested restcountirwes keys, if it is a list, join into a string of comma separated.
+- [X] Update user agent headers for get requests, rotate headers.
 
 Future Additions
 ----------------
@@ -368,4 +378,5 @@ Future Additions
 - [ ] Some sort of flag when creating a new class of the sw indicating that custom data is present in the object.
 - [ ] state_city_data parameter in get_all script that gets all the city data per subdivision from Country State City API using get request.
 - [X] Remove type Errors and just force data attribute types.
+- [ ] Can I integrate AI with app?
 <!-- https://github.com/smucode/react-world-flags -->
diff --git a/docs/api.rst b/docs/api.rst
@@ -8,7 +8,7 @@ API
 .. subdivision name and code can be used to search for a sought subdivision.
 
 The main API endpoint is: `https://iso3166-2-api.vercel.app/api <https://iso3166-2-api.vercel.app/api/>`_. This endpoint displays the API documentation and forms the
-base URL for the 5 other endpoints.
+base URL for the 6 other endpoints.
 
 The other endpoints available in the API are:
 
@@ -17,8 +17,9 @@ The other endpoints available in the API are:
 * https://iso3166-2-api.vercel.app/api/subdivision/<input_subdivision>
 * https://iso3166-2-api.vercel.app/api/country_name/<input_country_name> 
 * https://iso3166-2-api.vercel.app/api/name/<input_name>
+* https://iso3166-2-api.vercel.app/api/list_subdivisions
 
-Five paths/endpoints are available in the API - `/api/all`, `/api/alpha`, `/api/subdivision`, `/api/country_name` and `/api/name`.
+Five paths/endpoints are available in the API - `/api/all`, `/api/alpha`, `/api/subdivision`, `/api/country_name`, `/api/name` and `/api/list_subdivisions`.
 
 * `/api/all`: get all of the ISO 3166 subdivision data for all countries.
 
@@ -28,8 +29,10 @@ Five paths/endpoints are available in the API - `/api/all`, `/api/alpha`, `/api/
 
 * `/api/subdivision`: get all of the ISO 3166 subdivision data for 1 or more ISO 3166-2 subdivision codes, e.g `/api/subdivision/GB-ABD`. You can also input a comma separated list of subdivision codes from the same and or different countries and the data for each will be returned e.g `/api/subdivision/IE-MO,FI-17,RO-AG`. If the input subdivision code is not in the correct format then an error will be raised. Similarly if an invalid subdivision code that doesn't exist is input then an error will be raised.
 
-* `/api/name/`: get all of the ISO 3166 subdivision data for 1 or more ISO 3166-2 subdivision names, e.g `/api/name/Derry`. You can also input a comma separated list of subdivision name from the same or different countries and the data for each will be returned e.g `/api/name/Paris,Frankfurt,Rimini`. A closeness function is utilized to find the matching subdivision name, if no exact name match found then the most approximate subdivisions will be returned. Some subdivisions may have the same name, in this case each subdivision and its data will be returned e.g `/api/name/Saint George` (this example returns 5 subdivisions). This endpoint also has the likeness score (`?likeness=`) query string parameter that can be appended to the URL. This can be set between 1 - 100, representing a % of likeness to the input name the return subdivisions should be, e.g: a likeness score of 90 will return fewer potential matches whose name only match to a high degree compared to a score of 10 which will create a larger search space, thus returning more potential subdivision matches. A default likeness of 100 (exact match) is used, if no matching subdivision is found then this is reduced to 90. If an invalid subdivision name that doesn't match any is input then an error will be raised.
+* `/api/name`: get all of the ISO 3166 subdivision data for 1 or more ISO 3166-2 subdivision names, e.g `/api/name/Derry`. You can also input a comma separated list of subdivision name from the same or different countries and the data for each will be returned e.g `/api/name/Paris,Frankfurt,Rimini`. A closeness function is utilized to find the matching subdivision name, if no exact name match found then the most approximate subdivisions will be returned. Some subdivisions may have the same name, in this case each subdivision and its data will be returned e.g `/api/name/Saint George` (this example returns 5 subdivisions). This endpoint also has the likeness score (`?likeness=`) query string parameter that can be appended to the URL. This can be set between 1 - 100, representing a % of likeness to the input name the return subdivisions should be, e.g: a likeness score of 90 will return fewer potential matches whose name only match to a high degree compared to a score of 10 which will create a larger search space, thus returning more potential subdivision matches. A default likeness of 100 (exact match) is used, if no matching subdivision is found then this is reduced to 90. If an invalid subdivision name that doesn't match any is input then an error will be raised.
 
+* `/api/list_subdivisions`: get list of all subdivision codes per country.
+
 * `/api`: main homepage and API documentation.
 
 Get subdivision data for all countries
@@ -223,4 +226,25 @@ curl::
 
 .. |demo_link| raw:: html
 
-   <a href="https://colab.research.google.com/drive/1btfEx23bgWdkUPiwdwlDqKkmUp1S-_7U?usp=sharing" target="_blank">here</a>
+   <a href="https://colab.research.google.com/drive/1btfEx23bgWdkUPiwdwlDqKkmUp1S-_7U?usp=sharing" target="_blank">here</a>
+
+Get list of all subdivision codes per country
+---------------------------------------------
+Return a list of all ISO 3166-2 subdivision codes for each country.
+
+Python Requests:
+
+.. code-block:: python
+
+    import requests
+
+    base_url = "https://iso3166-2-api.vercel.app/api/list_subdivisions"
+    all_data = requests.get(base_url)
+
+    all_data["DE"] #subdivision codes for Germany
+    all_data["OM"] #subdivision data for Oman
+    all_data["US"] #subdivision data for US
+
+curl::
+
+    $ curl -i https://iso3166-2-api.vercel.app/api/list_subdivisions
diff --git a/docs/index.rst b/docs/index.rst
@@ -45,12 +45,12 @@ There are 7 main data attributes available for each subdivision within the **iso
 
 Last Updated
 ============
-The ISO 3166-2 data was last updated on March 2023. A log of the latest ISO 3166-2 updates can be seen in the 
+The ISO 3166-2 data was last updated on **March 2024**. A log of the latest ISO 3166-2 updates can be seen in the 
 |updates_md_link| file in the repository.
 
 .. |updates_md_link| raw:: html
 
-   <a href="https://github.com/amckenna41/iso3166-updates" target="_blank">UPDATES.md</a>
+   <a href="https://github.com/amckenna41/iso3166-updates/blob/main/UPDATES.MD" target="_blank">UPDATES.md</a>
 
 .. |license_repo_link| raw:: html
 

diff --git a/iso3166_2/__init__.py b/iso3166_2/__init__.py
@@ -2,7 +2,7 @@
 
 #software metadata
 __name__ = 'iso3166-2'
-__version__ = "1.6.0"
+__version__ = "1.6.1"
 __description__ = "A lightweight Python package, and accompanying API, used to access all of the world's most up-to-date and accurate ISO 3166-2 subdivision data, including: name, local name, code, parent code, type, latitude/longitude and flag."
 __author__ = 'AJ McKenna, https://github.com/amckenna41'
 __authorEmail__ = 'amckenna41@qub.ac.uk'