{ "cells": [ { "cell_type": "markdown", "metadata": { "cell_id": "00000-a14710bb-0646-4bbc-bd8f-38596c0b34f2", "deepnote_cell_type": "markdown" }, "source": [ "# Wind Farms Dataset Quick Start \n", "\n", "Version 0.9\n", "\n", "The Wind Farms is an Academic Hub dataset hosted by the OSIsoft Cloud Service (OCS, https://www.osisoft.com/solutions/cloud/vision/), a cloud-native real-time data infrastructure used to perform enterprise-wide analytics using tools and languages of the user's choice. \n", "\n", "
\n", "For documentation about the Wind Farms dataset itself, please go to this link on academic notebook viewer.\n", "
\n", "\n", "**Raw operational data has specific characteristics making it difficult to deal with directly**, among them:\n", "\n", "* variable data collection frequencies\n", "* bad values (system error codes)\n", "* data gaps \n", "\n", "\n", "**But data science projects using operational data needs to be:**\n", "\n", "* **Time-aligned** to deal with the characteritics above in consistent way according to the data type (e.g. interpolation for float values, repeat last good value for categorical data, etc)\n", "* **Context aware** so that the data can be understandable, across as many real-world assets that you need it for\n", "* **Shaped and filtered** to ensure you have the data you need, in the form you need it\n", "\n", "**The OCS solutions for application-ready data are Data Views:**\n", "\n", "![](https://academichub.blob.core.windows.net/images/piworld-dse-dataview-p2.png)\n", "\n", "**Each Academic Hub datasets comes endowed with a set of asset-centric data views.** \n", "\n", "The goal of Academic Hub Python library is to provide a very generic and consistent way to access:\n", "\n", "* the list of existing datasets\n", "* for a given dataset:\n", " * the list of its assets\n", " * the OCS namespace where the dataset is hosted\n", "* for a given asset, the list of data views it belongs to\n", "\n", "
\n", "The rest of this notebook is a working example of the functionality listed above for the Wind Farms dataset\n", "
\n" ] }, { "cell_type": "markdown", "metadata": { "cell_id": "00001-95c9d03b-be21-49f3-bd77-d810b012d652", "deepnote_cell_type": "markdown" }, "source": [ "## Install Academic Hub Python library " ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "cell_id": "00002-66950a56-8cfe-4c5e-9044-e7b80abd14c3", "deepnote_cell_type": "code", "deepnote_to_be_reexecuted": false, "execution_millis": 2785, "execution_start": 1620772990164, "source_hash": "5264110c", "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Requirement already satisfied: ocs-academic-hub==0.97.0 in c:\\users\\cfoisy\\anaconda3\\envs\\datapump\\lib\\site-packages (0.97.0)\n", "Requirement already satisfied: pandas>=0.24.2 in c:\\users\\cfoisy\\anaconda3\\envs\\datapump\\lib\\site-packages (from ocs-academic-hub==0.97.0) (1.1.4)\n", "Requirement already satisfied: python-dateutil>=2.8.0 in c:\\users\\cfoisy\\anaconda3\\envs\\datapump\\lib\\site-packages (from ocs-academic-hub==0.97.0) (2.8.1)\n", "Requirement already satisfied: numpy in c:\\users\\cfoisy\\anaconda3\\envs\\datapump\\lib\\site-packages (from ocs-academic-hub==0.97.0) (1.19.4)\n", "Requirement already satisfied: backoff in c:\\users\\cfoisy\\anaconda3\\envs\\datapump\\lib\\site-packages (from ocs-academic-hub==0.97.0) (1.10.0)\n", "Requirement already satisfied: gql in c:\\users\\cfoisy\\anaconda3\\envs\\datapump\\lib\\site-packages (from ocs-academic-hub==0.97.0) (2.0.0)\n", "Requirement already satisfied: typeguard>=2.4.1 in c:\\users\\cfoisy\\anaconda3\\envs\\datapump\\lib\\site-packages (from ocs-academic-hub==0.97.0) (2.10.0)\n", "Requirement already satisfied: ocs-sample-library-hub==0.1.18 in c:\\users\\cfoisy\\anaconda3\\envs\\datapump\\lib\\site-packages (from ocs-academic-hub==0.97.0) (0.1.18)\n", "Requirement already satisfied: pytz>=2017.2 in c:\\users\\cfoisy\\anaconda3\\envs\\datapump\\lib\\site-packages (from pandas>=0.24.2->ocs-academic-hub==0.97.0) (2020.4)\n", "Requirement already satisfied: six>=1.5 in c:\\users\\cfoisy\\anaconda3\\envs\\datapump\\lib\\site-packages (from python-dateutil>=2.8.0->ocs-academic-hub==0.97.0) (1.15.0)\n", "Requirement already satisfied: promise<3,>=2.3 in c:\\users\\cfoisy\\anaconda3\\envs\\datapump\\lib\\site-packages (from gql->ocs-academic-hub==0.97.0) (2.3)\n", "Requirement already satisfied: graphql-core<3,>=2.3.2 in c:\\users\\cfoisy\\anaconda3\\envs\\datapump\\lib\\site-packages (from gql->ocs-academic-hub==0.97.0) (2.3.2)\n", "Requirement already satisfied: requests<3,>=2.12 in c:\\users\\cfoisy\\anaconda3\\envs\\datapump\\lib\\site-packages (from gql->ocs-academic-hub==0.97.0) (2.25.1)\n", "Requirement already satisfied: rx<2,>=1.6 in c:\\users\\cfoisy\\anaconda3\\envs\\datapump\\lib\\site-packages (from graphql-core<3,>=2.3.2->gql->ocs-academic-hub==0.97.0) (1.6.1)\n", "Requirement already satisfied: certifi>=2017.4.17 in c:\\users\\cfoisy\\anaconda3\\envs\\datapump\\lib\\site-packages (from requests<3,>=2.12->gql->ocs-academic-hub==0.97.0) (2020.11.8)\n", "Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\\users\\cfoisy\\anaconda3\\envs\\datapump\\lib\\site-packages (from requests<3,>=2.12->gql->ocs-academic-hub==0.97.0) (1.25.11)\n", "Requirement already satisfied: chardet<5,>=3.0.2 in c:\\users\\cfoisy\\anaconda3\\envs\\datapump\\lib\\site-packages (from requests<3,>=2.12->gql->ocs-academic-hub==0.97.0) (3.0.4)\n", "Requirement already satisfied: idna<3,>=2.5 in c:\\users\\cfoisy\\anaconda3\\envs\\datapump\\lib\\site-packages (from requests<3,>=2.12->gql->ocs-academic-hub==0.97.0) (2.10)\n" ] } ], "source": [ "!pip install ocs-academic-hub==0.97.0" ] }, { "cell_type": "markdown", "metadata": { "cell_id": "00003-97d5b8a0-24aa-434b-a318-bc71f3aa98a4", "deepnote_cell_type": "markdown" }, "source": [ "## [Optional] Use the `pip uninstall` only in case of library issues" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "cell_id": "00004-0fe2654d-2708-4d0a-8b2c-1070b0718613", "deepnote_cell_type": "code", "deepnote_to_be_reexecuted": false, "execution_millis": 2, "execution_start": 1620772992953, "source_hash": "93345d6b", "tags": [] }, "outputs": [], "source": [ "# It's sometimes necessary to uninstall previous versions, uncomment and run the following line. \n", "# Then restart kernel and reinstall with previous cell\n", "# !pip uninstall -y ocs-academic-hub ocs-sample-library-preview" ] }, { "cell_type": "markdown", "metadata": { "cell_id": "00005-9db2e4b1-a323-4fcc-b61c-9ceb15717f5e", "deepnote_cell_type": "markdown" }, "source": [ "## Import HubClient, necessary to connect and interact with OCS" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "cell_id": "00006-9573ed9d-c2e7-4ba6-b14c-9688127142af", "deepnote_cell_type": "code", "deepnote_to_be_reexecuted": false, "execution_millis": 451, "execution_start": 1620772992960, "source_hash": "3e9edd1a" }, "outputs": [], "source": [ "from ocs_academic_hub import HubClient" ] }, { "cell_type": "markdown", "metadata": { "cell_id": "00007-6e6d1059-e493-4559-882b-91bd5cea010b", "deepnote_cell_type": "markdown" }, "source": [ "## Running the following cell initiate the login sequence\n", "\n", "**Warning:** a new brower tab will open offering the choice of identifying with Microsoft or Google. You should always pick Google:\n", "\"Login\n", "\n", "Return to this web page when done" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "cell_id": "00008-ba3deb68-cabb-4417-bc67-41ff9ca27aff", "deepnote_cell_type": "code", "deepnote_to_be_reexecuted": false, "execution_millis": 386, "execution_start": 1620772993415, "source_hash": "6437a418", "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Step 1: Get OAuth endpoint configuration...\n", "Step 2: Set up server to process authorization response...\n", "Step 3: Authorize the user...\n", "Step 4: Set server to handle one request...\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "127.0.0.1 - - [11/May/2021 19:12:18] \"GET /callback.html?code=A09BB4059EF2838A36448D6888253383E39C213B90534CE2377A26297234C983&scope=openid%20ocsapi&session_state=nYKcABwOYHuOagIpY0PLBsmaf4rujWx9LeMlpfsNxrE.F226FA877176F741425BE4FD0A40B17D HTTP/1.1\" 200 -\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Step 5: Get a token using the authorization code...\n", "Step 6: Access token read ok\n", "Complete!\n", "@ Hub data file: hub_datasets.json\n" ] } ], "source": [ "# remove env line before publication\n", "# %env OCS_HUB_CONFIG=config.txt\n", "hub = HubClient()" ] }, { "cell_type": "markdown", "metadata": { "cell_id": "00009-8b31a611-80e1-402f-882b-d2bd8c259ea1", "deepnote_cell_type": "markdown" }, "source": [ "## Refresh datasets information\n", "\n", "Over time existing datasets are updated and new ones are added. The cell below makes sure you have the latest version of the production datasets. \n", "\n", "Note: after execution of this method, a file named `hub_datasets.json` will be created in the same directory as this notebook. The data in this file supersedes the one built-in with the `ocs_academic_hub` module. To get back to the built-in datasets information, move/rename/delete `hub_datasets.json`. " ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "cell_id": "00010-0c8d6f40-0ba6-4e21-af97-8a65af41b828", "deepnote_cell_type": "code", "deepnote_to_be_reexecuted": false, "execution_millis": 4613, "execution_start": 1620772993806, "source_hash": "dd92b3c8", "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "@ Hub data file: hub_datasets.json\n", "@ Current dataset: Brewery\n" ] } ], "source": [ "# remove experiment+additional_status options before publication\n", "hub.refresh_datasets(experimental=True, additional_status=\"agl\")" ] }, { "cell_type": "markdown", "metadata": { "cell_id": "00011-38f6136e-9843-4995-885b-4f5f00eb127c", "deepnote_cell_type": "markdown" }, "source": [ "## Get list of published hub datasets\n" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "cell_id": "00012-becfb7c1-bf51-499f-a887-c5e680b5d0a9", "deepnote_cell_type": "code", "deepnote_to_be_reexecuted": false, "execution_millis": 42, "execution_start": 1620772998420, "source_hash": "83f8b25d" }, "outputs": [ { "data": { "text/plain": [ "['Brewery', 'Campus_Energy', 'Pilot_Plant', 'Wind_Farms']" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "hub.datasets()" ] }, { "cell_type": "markdown", "metadata": { "cell_id": "00013-94f8ff34-a0a0-4771-a464-737db06ee1ec", "deepnote_cell_type": "markdown" }, "source": [ "## Display current active dataset\n", "\n", "The default dataset is Brewery. Only one dataset can be active. " ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "cell_id": "00014-45dbe114-ced2-4319-8e7d-87298b3e4407", "deepnote_cell_type": "code", "deepnote_to_be_reexecuted": false, "execution_millis": 41, "execution_start": 1620772998427, "source_hash": "5cd9c7fc" }, "outputs": [ { "data": { "text/plain": [ "'Brewery'" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "hub.current_dataset()" ] }, { "cell_type": "markdown", "metadata": { "cell_id": "00015-ca2438cc-5073-4f80-a9fa-f2473f030941", "deepnote_cell_type": "markdown" }, "source": [ "## Set Wind Farms as the current dataset" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "cell_id": "00016-2d13986f-0227-4a36-af21-986354769172", "deepnote_cell_type": "code", "deepnote_to_be_reexecuted": false, "execution_millis": 0, "execution_start": 1620772998439, "source_hash": "796a3a8e" }, "outputs": [], "source": [ "hub.set_dataset(\"Wind_Farms\")" ] }, { "cell_type": "markdown", "metadata": { "cell_id": "00017-c7aecf19-0723-482c-b038-e6501c1f027b", "deepnote_cell_type": "markdown" }, "source": [ "## Verify that Wind Farms is active" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "cell_id": "00018-49f7c915-fcae-4c94-b387-d259bc7fba8f", "deepnote_cell_type": "code", "deepnote_to_be_reexecuted": false, "execution_millis": 7, "execution_start": 1620772998480, "source_hash": "5cd9c7fc" }, "outputs": [ { "data": { "text/plain": [ "'Wind_Farms'" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "hub.current_dataset()" ] }, { "cell_type": "markdown", "metadata": { "cell_id": "00019-e29188ad-2dbc-4fd4-adee-42c0fa376d5f", "deepnote_cell_type": "markdown" }, "source": [ "## Get list of assets with Data Views\n", "\n", "Returned into the form of a pandas dataframe, with column `Asset_Id` and `Description`. Each asset has a unique `Asset_Id` as its identity. \n", "\n", "Note that the asset *cluster1.turb2* is having index 2 (first column). We'll use this information in a few cells. " ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "cell_id": "00020-2fa05fc3-7ef4-4c83-b75b-bc97954b2a3a", "deepnote_cell_type": "code", "deepnote_to_be_reexecuted": false, "execution_millis": 102, "execution_start": 1620772998481, "source_hash": "300c2fb8", "tags": [] }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Asset_IdDescription
0cluster1.turb1Turbine
1cluster1.turb10Turbine
2cluster1.turb2Turbine
3cluster1.turb3Turbine
4cluster1.turb4Turbine
5cluster1.turb5Turbine
6cluster1.turb6Turbine
7cluster1.turb7Turbine
8cluster1.turb8Turbine
9cluster1.turb9Turbine
10cluster2.turb1Turbine
11cluster2.turb10Turbine
12cluster2.turb2Turbine
13cluster2.turb3Turbine
14cluster2.turb4Turbine
15cluster2.turb5Turbine
16cluster2.turb6Turbine
17cluster2.turb7Turbine
18cluster2.turb8Turbine
19cluster2.turb9Turbine
20cluster3.turb1Turbine
21cluster3.turb10Turbine
22cluster3.turb2Turbine
23cluster3.turb3Turbine
24cluster3.turb4Turbine
25cluster3.turb5Turbine
26cluster3.turb6Turbine
27cluster3.turb7Turbine
28cluster3.turb8Turbine
29cluster3.turb9Turbine
30cluster4.turb1Turbine
31cluster4.turb10Turbine
32cluster4.turb2Turbine
33cluster4.turb3Turbine
34cluster4.turb4Turbine
35cluster4.turb5Turbine
36cluster4.turb6Turbine
37cluster4.turb7Turbine
38cluster4.turb8Turbine
39cluster4.turb9Turbine
40cluster5.turb1Turbine
41cluster5.turb10Turbine
42cluster5.turb2Turbine
43cluster5.turb3Turbine
44cluster5.turb4Turbine
45cluster5.turb5Turbine
46cluster5.turb6Turbine
47cluster5.turb7Turbine
48cluster5.turb8Turbine
49cluster5.turb9Turbine
\n", "
" ], "text/plain": [ " Asset_Id Description\n", "0 cluster1.turb1 Turbine\n", "1 cluster1.turb10 Turbine\n", "2 cluster1.turb2 Turbine\n", "3 cluster1.turb3 Turbine\n", "4 cluster1.turb4 Turbine\n", "5 cluster1.turb5 Turbine\n", "6 cluster1.turb6 Turbine\n", "7 cluster1.turb7 Turbine\n", "8 cluster1.turb8 Turbine\n", "9 cluster1.turb9 Turbine\n", "10 cluster2.turb1 Turbine\n", "11 cluster2.turb10 Turbine\n", "12 cluster2.turb2 Turbine\n", "13 cluster2.turb3 Turbine\n", "14 cluster2.turb4 Turbine\n", "15 cluster2.turb5 Turbine\n", "16 cluster2.turb6 Turbine\n", "17 cluster2.turb7 Turbine\n", "18 cluster2.turb8 Turbine\n", "19 cluster2.turb9 Turbine\n", "20 cluster3.turb1 Turbine\n", "21 cluster3.turb10 Turbine\n", "22 cluster3.turb2 Turbine\n", "23 cluster3.turb3 Turbine\n", "24 cluster3.turb4 Turbine\n", "25 cluster3.turb5 Turbine\n", "26 cluster3.turb6 Turbine\n", "27 cluster3.turb7 Turbine\n", "28 cluster3.turb8 Turbine\n", "29 cluster3.turb9 Turbine\n", "30 cluster4.turb1 Turbine\n", "31 cluster4.turb10 Turbine\n", "32 cluster4.turb2 Turbine\n", "33 cluster4.turb3 Turbine\n", "34 cluster4.turb4 Turbine\n", "35 cluster4.turb5 Turbine\n", "36 cluster4.turb6 Turbine\n", "37 cluster4.turb7 Turbine\n", "38 cluster4.turb8 Turbine\n", "39 cluster4.turb9 Turbine\n", "40 cluster5.turb1 Turbine\n", "41 cluster5.turb10 Turbine\n", "42 cluster5.turb2 Turbine\n", "43 cluster5.turb3 Turbine\n", "44 cluster5.turb4 Turbine\n", "45 cluster5.turb5 Turbine\n", "46 cluster5.turb6 Turbine\n", "47 cluster5.turb7 Turbine\n", "48 cluster5.turb8 Turbine\n", "49 cluster5.turb9 Turbine" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from IPython.display import display, Markdown\n", "turbines = hub.assets()\n", "display(turbines)" ] }, { "cell_type": "markdown", "metadata": { "cell_id": "00021-e0724515-877b-456f-a678-6c881c4e0b1c", "deepnote_cell_type": "markdown" }, "source": [ "## List of all Data Views\n", "\n", "Those are all single-asset default (with all data available for the asset) Data Views" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "cell_id": "00022-d1d73201-29ce-4a3a-9968-d3aba13b1571", "deepnote_cell_type": "code", "deepnote_to_be_reexecuted": false, "execution_millis": 10, "execution_start": 1620772998592, "source_hash": "32433344" }, "outputs": [ { "data": { "text/plain": [ "['wind.farms_cluster1.turb1',\n", " 'wind.farms_cluster1.turb10',\n", " 'wind.farms_cluster1.turb2',\n", " 'wind.farms_cluster1.turb3',\n", " 'wind.farms_cluster1.turb4',\n", " 'wind.farms_cluster1.turb5',\n", " 'wind.farms_cluster1.turb6',\n", " 'wind.farms_cluster1.turb7',\n", " 'wind.farms_cluster1.turb8',\n", " 'wind.farms_cluster1.turb9',\n", " 'wind.farms_cluster2.turb1',\n", " 'wind.farms_cluster2.turb10',\n", " 'wind.farms_cluster2.turb2',\n", " 'wind.farms_cluster2.turb3',\n", " 'wind.farms_cluster2.turb4',\n", " 'wind.farms_cluster2.turb5',\n", " 'wind.farms_cluster2.turb6',\n", " 'wind.farms_cluster2.turb7',\n", " 'wind.farms_cluster2.turb8',\n", " 'wind.farms_cluster2.turb9',\n", " 'wind.farms_cluster3.turb1',\n", " 'wind.farms_cluster3.turb10',\n", " 'wind.farms_cluster3.turb2',\n", " 'wind.farms_cluster3.turb3',\n", " 'wind.farms_cluster3.turb4',\n", " 'wind.farms_cluster3.turb5',\n", " 'wind.farms_cluster3.turb6',\n", " 'wind.farms_cluster3.turb7',\n", " 'wind.farms_cluster3.turb8',\n", " 'wind.farms_cluster3.turb9',\n", " 'wind.farms_cluster4.turb1',\n", " 'wind.farms_cluster4.turb10',\n", " 'wind.farms_cluster4.turb2',\n", " 'wind.farms_cluster4.turb3',\n", " 'wind.farms_cluster4.turb4',\n", " 'wind.farms_cluster4.turb5',\n", " 'wind.farms_cluster4.turb6',\n", " 'wind.farms_cluster4.turb7',\n", " 'wind.farms_cluster4.turb8',\n", " 'wind.farms_cluster4.turb9',\n", " 'wind.farms_cluster5.turb1',\n", " 'wind.farms_cluster5.turb10',\n", " 'wind.farms_cluster5.turb2',\n", " 'wind.farms_cluster5.turb3',\n", " 'wind.farms_cluster5.turb4',\n", " 'wind.farms_cluster5.turb5',\n", " 'wind.farms_cluster5.turb6',\n", " 'wind.farms_cluster5.turb7',\n", " 'wind.farms_cluster5.turb8',\n", " 'wind.farms_cluster5.turb9']" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "hub.asset_dataviews()" ] }, { "cell_type": "markdown", "metadata": { "cell_id": "00023-24dac889-b8a9-4b4d-83a2-5fbdf864ae01", "deepnote_cell_type": "markdown" }, "source": [ "## List of Data Views exclusive to turbine `cluster3.turb2` \n", "\n", "Empty filter (`filter=\"\"`) allows to see all dataviews for the asset instead of simply the default one\n", "\n", "**NOTE: Turbines for `Wind_Farms` dataset have only the default data view**" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "cell_id": "00024-af050a48-6b13-4bb9-a717-a11b1a6ba224", "deepnote_cell_type": "code", "deepnote_to_be_reexecuted": false, "execution_millis": 9, "execution_start": 1620772998593, "source_hash": "7ef808dd" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Turbine Id: cluster3.turb2\n" ] }, { "data": { "text/plain": [ "['wind.farms_cluster3.turb2']" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "turbine_id = \"cluster3.turb2\"\n", "print(\"Turbine Id:\", turbine_id)\n", "dvs_turbine = hub.asset_dataviews(asset=turbine_id, filter=\"\")\n", "dvs_turbine" ] }, { "cell_type": "markdown", "metadata": { "cell_id": "00025-af27835d-a8e9-4f6d-9df0-a9ff310121b6", "deepnote_cell_type": "markdown" }, "source": [ "## Get the OCS namespace associated to the dataset\n", "\n", "Each data set belongs to a namespace within the Academic Hub OCS account. Since dataset may move over time, the function below always return the active namespace for the given dataset. " ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "cell_id": "00026-f86fd16d-95ec-4fc6-a0fd-8e70f1edd207", "deepnote_cell_type": "code", "deepnote_to_be_reexecuted": false, "execution_millis": 9, "execution_start": 1620772998595, "source_hash": "7a29829c" }, "outputs": [ { "data": { "text/plain": [ "'academic_hub_01'" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dataset = hub.current_dataset()\n", "namespace_id = hub.namespace_of(dataset)\n", "namespace_id" ] }, { "cell_type": "markdown", "metadata": { "cell_id": "00027-dbe01042-1102-4da6-9198-34fee704074b", "deepnote_cell_type": "markdown" }, "source": [ "## Get Data View structure\n", "\n", "With Stream Name, the column name under which stream data appears, its value type and engineering units if available. We display below the structure of the default data view. " ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "cell_id": "00028-4159e120-6845-4410-86bc-d2bafecc1da1", "deepnote_cell_type": "code", "deepnote_to_be_reexecuted": false, "execution_millis": 566, "execution_start": 1620772998598, "source_hash": "e76d9d53", "tags": [] }, "outputs": [ { "data": { "text/markdown": [ "**Structure of Data view ID** `wind.farms_cluster3.turb2` :" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Asset_IdColumn_NameStream_TypeStream_UOMOCS_Stream_Name
4cluster3.turb2Ambient TemperatureFloat°Ccluster3.turb2.temp_ambient
5cluster3.turb2Drivetrain Gearbox Temp IMSDEFloat°Ccluster3.turb2.temp_drivetrain_gearbox_IMSDE
6cluster3.turb2Drivetrain Gearbox Temp IMSNDEFloat°Ccluster3.turb2.temp_drivetrain_gearbox_IMSNDE
7cluster3.turb2Drivetrain Mainbearing TempFloat°Ccluster3.turb2.temp_drivetrain_mainbearing
9cluster3.turb2Drivetrain vibrationFloatm/s²cluster3.turb2.vib_drive_train
8cluster3.turb2Nacelle TempFloat°Ccluster3.turb2.temp_nacelle
1cluster3.turb2Pitch AngleFloatdegreescluster3.turb2.pitch_angle
2cluster3.turb2Power To GridFloatkWcluster3.turb2.power_to_grid
10cluster3.turb2Relative Wind DirectionFloatdegreescluster3.turb2.wind_direction_relative
3cluster3.turb2Rotor SpeedFloatRPMcluster3.turb2.rotor_rpm
0cluster3.turb2StateCategoryOK|TurbErrorcluster3.turb2.error_state
11cluster3.turb2Wind SpeedFloatm/scluster3.turb2.wind_speed
12cluster3.turb2Yaw AngleFloatdegreescluster3.turb2.yaw_angle
\n", "
" ], "text/plain": [ " Asset_Id Column_Name Stream_Type Stream_UOM \\\n", "4 cluster3.turb2 Ambient Temperature Float °C \n", "5 cluster3.turb2 Drivetrain Gearbox Temp IMSDE Float °C \n", "6 cluster3.turb2 Drivetrain Gearbox Temp IMSNDE Float °C \n", "7 cluster3.turb2 Drivetrain Mainbearing Temp Float °C \n", "9 cluster3.turb2 Drivetrain vibration Float m/s² \n", "8 cluster3.turb2 Nacelle Temp Float °C \n", "1 cluster3.turb2 Pitch Angle Float degrees \n", "2 cluster3.turb2 Power To Grid Float kW \n", "10 cluster3.turb2 Relative Wind Direction Float degrees \n", "3 cluster3.turb2 Rotor Speed Float RPM \n", "0 cluster3.turb2 State Category OK|TurbError \n", "11 cluster3.turb2 Wind Speed Float m/s \n", "12 cluster3.turb2 Yaw Angle Float degrees \n", "\n", " OCS_Stream_Name \n", "4 cluster3.turb2.temp_ambient \n", "5 cluster3.turb2.temp_drivetrain_gearbox_IMSDE \n", "6 cluster3.turb2.temp_drivetrain_gearbox_IMSNDE \n", "7 cluster3.turb2.temp_drivetrain_mainbearing \n", "9 cluster3.turb2.vib_drive_train \n", "8 cluster3.turb2.temp_nacelle \n", "1 cluster3.turb2.pitch_angle \n", "2 cluster3.turb2.power_to_grid \n", "10 cluster3.turb2.wind_direction_relative \n", "3 cluster3.turb2.rotor_rpm \n", "0 cluster3.turb2.error_state \n", "11 cluster3.turb2.wind_speed \n", "12 cluster3.turb2.yaw_angle " ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "dataview_id = hub.asset_dataviews(asset=turbine_id, filter=\"default\")[0]\n", "display(Markdown(f\"**Structure of Data view ID** `{dataview_id}` :\"))\n", "display(hub.dataview_definition(namespace_id, dataview_id))" ] }, { "cell_type": "markdown", "metadata": { "cell_id": "00029-e966fe2e-1def-4cd1-9cde-c8cc2b479152", "deepnote_cell_type": "markdown" }, "source": [ "## Getting data from a Data View\n", "\n", "Return interpolated data between a start and end date, with the requested interpolation interval (format is HH:MM:SS)" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "cell_id": "00030-7ef01ad6-203a-4670-bdb7-bfd0f603d955", "deepnote_cell_type": "code", "deepnote_to_be_reexecuted": false, "execution_millis": 9816, "execution_start": 1620772999168, "source_hash": "8c64a218", "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "+++++++\n", " ==> Finished 'dataview_interpolated_pd' in 10.9414 secs [ 4.08K rows/sec ]\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
TimestampAsset_IdPitch AnglePower To GridRotor SpeedAmbient TemperatureDrivetrain Gearbox Temp IMSDEDrivetrain Gearbox Temp IMSNDEDrivetrain Mainbearing TempNacelle TempDrivetrain vibrationRelative Wind DirectionWind SpeedYaw AngleState
02018-03-01 00:00:00cluster3.turb262.520000-0.0441510.010746-78.1398588.5523318.22331916.0156117.483289-0.018923-19.18505611.53671835.368730TurbError
12018-03-01 00:01:00cluster3.turb262.520000-0.0512220.011359-89.0763668.0723327.76178616.5663067.063290-0.018944-14.92871710.63519635.337961TurbError
22018-03-01 00:02:00cluster3.turb262.520000-0.0582930.011973-100.0128747.5923337.30025316.9046516.643291-0.018965-22.02394110.91455135.307192TurbError
32018-03-01 00:03:00cluster3.turb262.520000-0.0653650.012586-110.9493837.1123346.83872016.6320166.223292-0.018986-14.94100610.85480635.276423TurbError
42018-03-01 00:04:00cluster3.turb2561.566741-0.0724360.013200-121.8858916.6323356.37718716.3593825.803294-0.019007-23.41130411.34406035.245654TurbError
................................................
446362018-03-31 23:56:00cluster3.turb20.982178384.22623215.43125722.32545166.21290962.96010233.38904528.6742030.014219-1.4196216.533780279.834269OK
446372018-03-31 23:57:00cluster3.turb20.96946370.19626215.44290722.36366866.28349862.99478433.39154428.6888010.0140515.3659564.309831279.819859OK
446382018-03-31 23:58:00cluster3.turb20.956749136.62088815.45455822.40188466.35408663.02946633.39404328.7034000.0138839.5291204.877480279.805448OK
446392018-03-31 23:59:00cluster3.turb20.944034195.52366215.46620922.44010066.42467563.06414833.39654228.7179980.0137151.9819504.213840279.791038OK
446402018-04-01 00:00:00cluster3.turb20.273318355.45986515.47785922.47831766.49526363.09883133.39904028.7325970.013548-2.4933165.230510279.776627OK
\n", "

44641 rows × 15 columns

\n", "
" ], "text/plain": [ " Timestamp Asset_Id Pitch Angle Power To Grid \\\n", "0 2018-03-01 00:00:00 cluster3.turb2 62.520000 -0.044151 \n", "1 2018-03-01 00:01:00 cluster3.turb2 62.520000 -0.051222 \n", "2 2018-03-01 00:02:00 cluster3.turb2 62.520000 -0.058293 \n", "3 2018-03-01 00:03:00 cluster3.turb2 62.520000 -0.065365 \n", "4 2018-03-01 00:04:00 cluster3.turb2 561.566741 -0.072436 \n", "... ... ... ... ... \n", "44636 2018-03-31 23:56:00 cluster3.turb2 0.982178 384.226232 \n", "44637 2018-03-31 23:57:00 cluster3.turb2 0.969463 70.196262 \n", "44638 2018-03-31 23:58:00 cluster3.turb2 0.956749 136.620888 \n", "44639 2018-03-31 23:59:00 cluster3.turb2 0.944034 195.523662 \n", "44640 2018-04-01 00:00:00 cluster3.turb2 0.273318 355.459865 \n", "\n", " Rotor Speed Ambient Temperature Drivetrain Gearbox Temp IMSDE \\\n", "0 0.010746 -78.139858 8.552331 \n", "1 0.011359 -89.076366 8.072332 \n", "2 0.011973 -100.012874 7.592333 \n", "3 0.012586 -110.949383 7.112334 \n", "4 0.013200 -121.885891 6.632335 \n", "... ... ... ... \n", "44636 15.431257 22.325451 66.212909 \n", "44637 15.442907 22.363668 66.283498 \n", "44638 15.454558 22.401884 66.354086 \n", "44639 15.466209 22.440100 66.424675 \n", "44640 15.477859 22.478317 66.495263 \n", "\n", " Drivetrain Gearbox Temp IMSNDE Drivetrain Mainbearing Temp \\\n", "0 8.223319 16.015611 \n", "1 7.761786 16.566306 \n", "2 7.300253 16.904651 \n", "3 6.838720 16.632016 \n", "4 6.377187 16.359382 \n", "... ... ... \n", "44636 62.960102 33.389045 \n", "44637 62.994784 33.391544 \n", "44638 63.029466 33.394043 \n", "44639 63.064148 33.396542 \n", "44640 63.098831 33.399040 \n", "\n", " Nacelle Temp Drivetrain vibration Relative Wind Direction \\\n", "0 7.483289 -0.018923 -19.185056 \n", "1 7.063290 -0.018944 -14.928717 \n", "2 6.643291 -0.018965 -22.023941 \n", "3 6.223292 -0.018986 -14.941006 \n", "4 5.803294 -0.019007 -23.411304 \n", "... ... ... ... \n", "44636 28.674203 0.014219 -1.419621 \n", "44637 28.688801 0.014051 5.365956 \n", "44638 28.703400 0.013883 9.529120 \n", "44639 28.717998 0.013715 1.981950 \n", "44640 28.732597 0.013548 -2.493316 \n", "\n", " Wind Speed Yaw Angle State \n", "0 11.536718 35.368730 TurbError \n", "1 10.635196 35.337961 TurbError \n", "2 10.914551 35.307192 TurbError \n", "3 10.854806 35.276423 TurbError \n", "4 11.344060 35.245654 TurbError \n", "... ... ... ... \n", "44636 6.533780 279.834269 OK \n", "44637 4.309831 279.819859 OK \n", "44638 4.877480 279.805448 OK \n", "44639 4.213840 279.791038 OK \n", "44640 5.230510 279.776627 OK \n", "\n", "[44641 rows x 15 columns]" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# For a single month of data\n", "df_acad = hub.dataview_interpolated_pd(\n", " namespace_id, dataview_id, \"2018-03-01\", \"2018-04-01\", \"00:01:00\"\n", ")\n", "df_acad" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "cell_id": "00031-5214913b-78b4-4fd9-8ef5-bbef730147e4", "deepnote_cell_type": "code", "deepnote_to_be_reexecuted": false, "execution_millis": 22, "execution_start": 1620773008987, "source_hash": "303862fd", "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "RangeIndex: 44641 entries, 0 to 44640\n", "Data columns (total 15 columns):\n", " # Column Non-Null Count Dtype \n", "--- ------ -------------- ----- \n", " 0 Timestamp 44641 non-null datetime64[ns]\n", " 1 Asset_Id 44641 non-null object \n", " 2 Pitch Angle 44641 non-null float64 \n", " 3 Power To Grid 44641 non-null float64 \n", " 4 Rotor Speed 44641 non-null float64 \n", " 5 Ambient Temperature 44641 non-null float64 \n", " 6 Drivetrain Gearbox Temp IMSDE 44641 non-null float64 \n", " 7 Drivetrain Gearbox Temp IMSNDE 44641 non-null float64 \n", " 8 Drivetrain Mainbearing Temp 44641 non-null float64 \n", " 9 Nacelle Temp 44641 non-null float64 \n", " 10 Drivetrain vibration 44641 non-null float64 \n", " 11 Relative Wind Direction 44641 non-null float64 \n", " 12 Wind Speed 44641 non-null float64 \n", " 13 Yaw Angle 44641 non-null float64 \n", " 14 State 44641 non-null object \n", "dtypes: datetime64[ns](1), float64(12), object(2)\n", "memory usage: 5.1+ MB\n" ] } ], "source": [ "# Information about the dataframe - this is a Pandas operation \n", "df_acad.info()" ] }, { "cell_type": "markdown", "metadata": { "cell_id": "00032-90e60deb-4b42-41b6-b6dd-c638b00b03b6", "deepnote_cell_type": "markdown" }, "source": [ "## Asset metadata\n", "\n", "In some datasets like `Wind_Farms`, assets have metadata (static information) attached to them. This metadata comes in the form of a Python dictionary, i.e. a set of keys, each key with an associated value. The example below is representative of turbine metadata available with `Wind_Farms`. " ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "cell_id": "00033-0eb2c777-fcc7-4d4a-b9ad-8b24a09d02f9", "deepnote_cell_type": "code", "deepnote_to_be_reexecuted": false, "execution_millis": 10, "execution_start": 1620773009001, "source_hash": "96c94d39", "tags": [] }, "outputs": [ { "data": { "text/plain": [ "{'Cluster': 3,\n", " 'ID': '2',\n", " 'Latitude': -35.113458,\n", " 'Longitude': 137.719395,\n", " 'Manufacturer': '',\n", " 'Model': '',\n", " 'Asset_Id': 'cluster3.turb2'}" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "hub.asset_metadata(turbine_id)" ] }, { "cell_type": "markdown", "metadata": { "cell_id": "00034-077e86ae-35f2-4ac1-8de1-242841bd65fc", "deepnote_cell_type": "markdown" }, "source": [ "## Metadata for all assets\n", "\n", "It sometimes useful to get metadata of all assets into a single Pandas dataframe to select assets according to some criteria, for example Primary Usage. " ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "cell_id": "00035-ed5f48df-d02f-4b02-9d71-83b07c32efb4", "deepnote_cell_type": "code", "deepnote_to_be_reexecuted": false, "execution_millis": 30, "execution_start": 1620773009019, "source_hash": "fcb7082a" }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
ClusterIDLatitudeLongitudeManufacturerModelAsset_Id
411-38.016069142.139145cluster1.turb1
7110-38.008225142.172404cluster1.turb10
1312-38.018385142.143941cluster1.turb2
1513-38.017802142.149638cluster1.turb3
2314-38.021606142.153468cluster1.turb4
2615-38.015985142.152867cluster1.turb5
3016-38.013043142.156611cluster1.turb6
3517-38.011429142.160753cluster1.turb7
4118-38.008259142.162984cluster1.turb8
4619-38.008250142.167297cluster1.turb9
321-37.691776142.515474cluster2.turb1
9210-37.696055142.566875cluster2.turb10
1422-37.694349142.519175cluster2.turb2
1823-37.693415142.525033cluster2.turb3
2124-37.691072142.528681cluster2.turb4
2725-37.689841142.536266cluster2.turb5
3426-37.688041142.540386cluster2.turb6
3727-37.687931142.553743cluster2.turb7
4428-37.695758142.540890cluster2.turb8
4529-37.696420142.536513cluster2.turb9
231-35.111825137.722635cluster3.turb1
5310-35.119118137.728172cluster3.turb10
1032-35.113458137.719395cluster3.turb2
1733-35.113800137.728665cluster3.turb3
2234-35.115038137.716187cluster3.turb4
2835-35.115503137.725382cluster3.turb5
3236-35.116784137.712808cluster3.turb6
3837-35.117091137.722088cluster3.turb7
4238-35.117732137.730811cluster3.turb8
4839-35.118680137.718945cluster3.turb9
141-33.294081138.731074cluster4.turb1
6410-33.302959138.718843cluster4.turb10
1142-33.297077138.728607cluster4.turb2
1943-33.296198138.706698cluster4.turb3
2044-33.295929138.712857cluster4.turb4
2545-33.296198138.719079cluster4.turb5
3146-33.298906138.723414cluster4.turb6
3947-33.300430138.727212cluster4.turb7
4048-33.301506138.707042cluster4.turb8
4749-33.300915138.712792cluster4.turb9
051-31.835897141.235764cluster5.turb1
8510-31.804245141.248103cluster5.turb10
1252-31.829316141.239241cluster5.turb2
1653-31.825159141.239176cluster5.turb3
2454-31.821658141.240163cluster5.turb4
2955-31.820765141.242545cluster5.turb5
3356-31.820856141.246751cluster5.turb6
3657-31.813891141.241086cluster5.turb7
4358-31.811393141.242760cluster5.turb8
4959-31.810335141.245206cluster5.turb9
\n", "
" ], "text/plain": [ " Cluster ID Latitude Longitude Manufacturer Model Asset_Id\n", "4 1 1 -38.016069 142.139145 cluster1.turb1\n", "7 1 10 -38.008225 142.172404 cluster1.turb10\n", "13 1 2 -38.018385 142.143941 cluster1.turb2\n", "15 1 3 -38.017802 142.149638 cluster1.turb3\n", "23 1 4 -38.021606 142.153468 cluster1.turb4\n", "26 1 5 -38.015985 142.152867 cluster1.turb5\n", "30 1 6 -38.013043 142.156611 cluster1.turb6\n", "35 1 7 -38.011429 142.160753 cluster1.turb7\n", "41 1 8 -38.008259 142.162984 cluster1.turb8\n", "46 1 9 -38.008250 142.167297 cluster1.turb9\n", "3 2 1 -37.691776 142.515474 cluster2.turb1\n", "9 2 10 -37.696055 142.566875 cluster2.turb10\n", "14 2 2 -37.694349 142.519175 cluster2.turb2\n", "18 2 3 -37.693415 142.525033 cluster2.turb3\n", "21 2 4 -37.691072 142.528681 cluster2.turb4\n", "27 2 5 -37.689841 142.536266 cluster2.turb5\n", "34 2 6 -37.688041 142.540386 cluster2.turb6\n", "37 2 7 -37.687931 142.553743 cluster2.turb7\n", "44 2 8 -37.695758 142.540890 cluster2.turb8\n", "45 2 9 -37.696420 142.536513 cluster2.turb9\n", "2 3 1 -35.111825 137.722635 cluster3.turb1\n", "5 3 10 -35.119118 137.728172 cluster3.turb10\n", "10 3 2 -35.113458 137.719395 cluster3.turb2\n", "17 3 3 -35.113800 137.728665 cluster3.turb3\n", "22 3 4 -35.115038 137.716187 cluster3.turb4\n", "28 3 5 -35.115503 137.725382 cluster3.turb5\n", "32 3 6 -35.116784 137.712808 cluster3.turb6\n", "38 3 7 -35.117091 137.722088 cluster3.turb7\n", "42 3 8 -35.117732 137.730811 cluster3.turb8\n", "48 3 9 -35.118680 137.718945 cluster3.turb9\n", "1 4 1 -33.294081 138.731074 cluster4.turb1\n", "6 4 10 -33.302959 138.718843 cluster4.turb10\n", "11 4 2 -33.297077 138.728607 cluster4.turb2\n", "19 4 3 -33.296198 138.706698 cluster4.turb3\n", "20 4 4 -33.295929 138.712857 cluster4.turb4\n", "25 4 5 -33.296198 138.719079 cluster4.turb5\n", "31 4 6 -33.298906 138.723414 cluster4.turb6\n", "39 4 7 -33.300430 138.727212 cluster4.turb7\n", "40 4 8 -33.301506 138.707042 cluster4.turb8\n", "47 4 9 -33.300915 138.712792 cluster4.turb9\n", "0 5 1 -31.835897 141.235764 cluster5.turb1\n", "8 5 10 -31.804245 141.248103 cluster5.turb10\n", "12 5 2 -31.829316 141.239241 cluster5.turb2\n", "16 5 3 -31.825159 141.239176 cluster5.turb3\n", "24 5 4 -31.821658 141.240163 cluster5.turb4\n", "29 5 5 -31.820765 141.242545 cluster5.turb5\n", "33 5 6 -31.820856 141.246751 cluster5.turb6\n", "36 5 7 -31.813891 141.241086 cluster5.turb7\n", "43 5 8 -31.811393 141.242760 cluster5.turb8\n", "49 5 9 -31.810335 141.245206 cluster5.turb9" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df_metadata = hub.all_assets_metadata()\n", "df_metadata" ] }, { "cell_type": "markdown", "metadata": { "cell_id": "00036-c1d3beb2-605b-4319-8e46-0be325b82136", "deepnote_cell_type": "markdown", "tags": [] }, "source": [ "## Map of Cluster 3 Wind Turbines\n", "\n", "NOTE: requires a [Mapbox](https://www.mapbox.com/) token to rerun. " ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "cell_id": "00036-e81097cd-88c9-428b-9b42-3d58ad3ba5a8", "deepnote_cell_type": "code", "deepnote_to_be_reexecuted": false, "execution_millis": 1615, "execution_start": 1620773009047, "source_hash": "326c6999", "tags": [] }, "outputs": [ { "data": { "text/html": [ " \n", " " ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.plotly.v1+json": { "config": { "plotlyServerURL": "https://plot.ly" }, "data": [ { "hovertemplate": "Asset_Id=%{text}
Latitude=%{lat}
Longitude=%{lon}", "lat": [ -35.111825, -35.119118, -35.113458, -35.1138, -35.115038, -35.115503, -35.116784, -35.117091, -35.117732, -35.11868 ], "legendgroup": "", "lon": [ 137.722635, 137.728172, 137.719395, 137.728665, 137.716187, 137.725382, 137.712808, 137.722088, 137.730811, 137.718945 ], "marker": { "color": "blue" }, "mode": "markers+text", "name": "", "showlegend": false, "subplot": "mapbox", "text": [ "cluster3.turb1", "cluster3.turb10", "cluster3.turb2", "cluster3.turb3", "cluster3.turb4", "cluster3.turb5", "cluster3.turb6", "cluster3.turb7", "cluster3.turb8", "cluster3.turb9" ], "type": "scattermapbox" } ], "layout": { "legend": { "tracegroupgap": 0 }, "mapbox": { "accesstoken": "pk.eyJ1IjoiY2ZvaXN5IiwiYSI6ImNrNXBveGNyNTA4cm0zbXFvY2cyZ2tveDAifQ.7wf11z7BjtTgKjj8Z1TvMA", "center": { "lat": -35.115902899999995, "lon": 137.72250879999999 }, "domain": { "x": [ 0, 1 ], "y": [ 0, 1 ] }, "zoom": 12 }, "margin": { "t": 60 }, "template": { "data": { "bar": [ { "error_x": { "color": "#2a3f5f" }, "error_y": { "color": "#2a3f5f" }, "marker": { "line": { "color": "#E5ECF6", "width": 0.5 } }, "type": "bar" } ], "barpolar": [ { "marker": { "line": { "color": "#E5ECF6", "width": 0.5 } }, "type": "barpolar" } ], "carpet": [ { "aaxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "baxis": { "endlinecolor": "#2a3f5f", "gridcolor": "white", "linecolor": "white", "minorgridcolor": "white", "startlinecolor": "#2a3f5f" }, "type": "carpet" } ], "choropleth": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "choropleth" } ], "contour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "contour" } ], "contourcarpet": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "contourcarpet" } ], "heatmap": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmap" } ], "heatmapgl": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "heatmapgl" } ], "histogram": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "histogram" } ], "histogram2d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2d" } ], "histogram2dcontour": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "histogram2dcontour" } ], "mesh3d": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "type": "mesh3d" } ], "parcoords": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "parcoords" } ], "pie": [ { "automargin": true, "type": "pie" } ], "scatter": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter" } ], "scatter3d": [ { "line": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatter3d" } ], "scattercarpet": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattercarpet" } ], "scattergeo": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergeo" } ], "scattergl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattergl" } ], "scattermapbox": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scattermapbox" } ], "scatterpolar": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolar" } ], "scatterpolargl": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterpolargl" } ], "scatterternary": [ { "marker": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "type": "scatterternary" } ], "surface": [ { "colorbar": { "outlinewidth": 0, "ticks": "" }, "colorscale": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "type": "surface" } ], "table": [ { "cells": { "fill": { "color": "#EBF0F8" }, "line": { "color": "white" } }, "header": { "fill": { "color": "#C8D4E3" }, "line": { "color": "white" } }, "type": "table" } ] }, "layout": { "annotationdefaults": { "arrowcolor": "#2a3f5f", "arrowhead": 0, "arrowwidth": 1 }, "coloraxis": { "colorbar": { "outlinewidth": 0, "ticks": "" } }, "colorscale": { "diverging": [ [ 0, "#8e0152" ], [ 0.1, "#c51b7d" ], [ 0.2, "#de77ae" ], [ 0.3, "#f1b6da" ], [ 0.4, "#fde0ef" ], [ 0.5, "#f7f7f7" ], [ 0.6, "#e6f5d0" ], [ 0.7, "#b8e186" ], [ 0.8, "#7fbc41" ], [ 0.9, "#4d9221" ], [ 1, "#276419" ] ], "sequential": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ], "sequentialminus": [ [ 0, "#0d0887" ], [ 0.1111111111111111, "#46039f" ], [ 0.2222222222222222, "#7201a8" ], [ 0.3333333333333333, "#9c179e" ], [ 0.4444444444444444, "#bd3786" ], [ 0.5555555555555556, "#d8576b" ], [ 0.6666666666666666, "#ed7953" ], [ 0.7777777777777778, "#fb9f3a" ], [ 0.8888888888888888, "#fdca26" ], [ 1, "#f0f921" ] ] }, "colorway": [ "#636efa", "#EF553B", "#00cc96", "#ab63fa", "#FFA15A", "#19d3f3", "#FF6692", "#B6E880", "#FF97FF", "#FECB52" ], "font": { "color": "#2a3f5f" }, "geo": { "bgcolor": "white", "lakecolor": "white", "landcolor": "#E5ECF6", "showlakes": true, "showland": true, "subunitcolor": "white" }, "hoverlabel": { "align": "left" }, "hovermode": "closest", "mapbox": { "style": "light" }, "paper_bgcolor": "white", "plot_bgcolor": "#E5ECF6", "polar": { "angularaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "radialaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "scene": { "xaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "yaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" }, "zaxis": { "backgroundcolor": "#E5ECF6", "gridcolor": "white", "gridwidth": 2, "linecolor": "white", "showbackground": true, "ticks": "", "zerolinecolor": "white" } }, "shapedefaults": { "line": { "color": "#2a3f5f" } }, "ternary": { "aaxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "baxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" }, "bgcolor": "#E5ECF6", "caxis": { "gridcolor": "white", "linecolor": "white", "ticks": "" } }, "title": { "x": 0.05 }, "xaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 }, "yaxis": { "automargin": true, "gridcolor": "white", "linecolor": "white", "ticks": "", "title": { "standoff": 15 }, "zerolinecolor": "white", "zerolinewidth": 2 } } } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import plotly.express as px\n", "\n", "px.set_mapbox_access_token(open(\"mapbox_token.txt\").read())\n", "fig = px.scatter_mapbox(\n", " df_metadata[df_metadata[\"Cluster\"] == 3],\n", " lat=\"Latitude\",\n", " lon=\"Longitude\",\n", " text=\"Asset_Id\",\n", " color_discrete_sequence=[\"blue\", \"red\", \"yellow\", \"orange\", \"purple\"],\n", " zoom=12,\n", ")\n", "fig.show()" ] }, { "cell_type": "markdown", "metadata": { "created_in_deepnote_cell": true, "deepnote_cell_type": "markdown", "tags": [] }, "source": [ "\n", "Created in deepnote.com \n", "Created in Deepnote" ] } ], "metadata": { "deepnote": {}, "deepnote_execution_queue": [], "deepnote_notebook_id": "35620d82-d6c3-4f68-86de-a81e1df5c403", "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.9" }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": true, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": false, "toc_position": {}, "toc_section_display": true, "toc_window_display": false } }, "nbformat": 4, "nbformat_minor": 4 }