Tag: PySpark

We will show how to search for a dissolved company using Companies House Public API. Using Databricks to call the api and then creating dataframe using PySpark.

Steps

• call https://api.company-information.service.gov.uk/dissolved-search/companies with required parameters
• use json payload to create a dataframe

See How to get Companies House data using REST API

Code snippets

# Companies House API
def call_api(url,q='qwerty',index=0):
  """ getting data from Companies House using Public API
  """
  params = dict(
    q={q},
    search_type='best-match',
    start_index = {index},
    size=100  
  )
  response = requests.get(url,params=params,auth=(ch_api_key,''))
  payload = response.json()
  return payload

# get some data
import json
payload_json = call_api('https://api.company-information.service.gov.uk/dissolved-search/companies','s',101) 

# create dataframe
df =spark.createDataFrame(payload_json['items'])