Friday, April 13, 2018

IDF table from equation - Pandas


How to fill a table (pandas DataFrame) with rainfall intensity values, extracted from idf equation:


import pandas as pd

durs = [5,10,15,20,30,60,120,180,360,540,720,1440] # rainfall durations
rps=[5,10,25,50,100] # return periods


t1 = pd.DataFrame(index=rps,columns=durs) # initialize DataFrame with rp's and durations

cf = [942.76,.1242,17.0,.650] #example idf equation coefficients

table = t1.apply(lambda x: (cf[0]*x.index**cf[1])/((x.name+cf[2])**cf[3]))

Wednesday, February 28, 2018

Installing Python in Windows


WinPython is a free open-source portable distribution of the Python programming language for Windows 7/8/10 and scientific and educational usage.


Designed for scientists, data-scientists, and education.


WinPython is a portable application, so the user should not expect any integration into Windows explorer during installation. However, the WinPython Control Panel allows to "register" your distribution to Windows.


Free Download at:

Thursday, February 8, 2018

Daily precipitation data to monthly and yearly

With pandas, we can group datetime values according to datetime units, as months and years.

If we have a pandas dataframe with 2 columns  - DATE and VALUE

Certify that the column DATE is recognized as datetime64 format - you can use, for example:

df1['DATE'] = pd.to_datetime(df1['DATE'])

and then you can group and make operations on this group.

Getting the average of total monthly precipitation:

monthly = df1.groupby(df1['DATE'].dt.month).sum() / len(pd.unique(df1['DATE'].dt.year))

(groups by month sum and then divide by the number of years)

Getting the average yearly precipitation:

PAnual = df1.groupby(df1[0].dt.year).sum().mean()

Friday, January 26, 2018

Accumulating Precipitation Data

If we have a pandas dataframe named df1 with a column '15min' containing 15 minute precipitation data, we can easily accumulate for other durations, using the rolling method as shown in the example below:

df1['01 h'] =df1['15min'].rolling(window=4,center=False).sum()

Note that the window=4 parameter means that it will accumulate 4 lines of 15 minute data, resulting in 1 hour precipitation. This parameter can be changed to whatever duration you want.

Tuesday, January 23, 2018

Polynomial Curve Fitting

The code below shows how easily you can do a Polynomial Curve Fitting with Python and Numpy.

import numpy as np

# sample x and y data - example
x = [7.76,10.11,11.89,14.81,15.49]
y = [1.851,1.971,1.953,1.842,1.805]

# the polyfit functions does the nth degree polynomial best fit on the data, 
# returning the polynomial coefficients

n = 4   # 4th degree polynomial, you can change for whatever degree you want
coefs = np.polyfit(x,y,n)

# The poly1d function applies the polynomial function to our calculated coefficients
polyf = np.poly1d(coefs)

#if we want to apply our polynomial function to a range of x values
xf = np.linspace(0,20)
yf = polyf(xf)

Sunday, November 26, 2017

Pandas sum column values according to another columns value


One-liner code to sum Pandas second columns according to same values in the first column.

df2 = df1.groupby(df1.columns[0])[df1.columns[1]].sum().reset_index()

For example, applying to a table listing pipe diameters and lenghts, the command will return total lenghts according to each unique diameters.

This functionality is similar to excel's pivot table sum.

Wednesday, November 22, 2017

Matplotlib template for Precipitation and Flow over time - Hidrograms and Hyetographs

This code works in python3, to plot a hyetograph - precipitation event (sP), together with a hydrograph (sQ), both in the same time series (t).

#-*-coding:utf-8-*-;
import matplotlib.pyplot as plt
from matplotlib import gridspec
import numpy as np

def plotHH(t,sP,sQ):
    fig = plt.figure()
    gs = gridspec.GridSpec(2, 1, height_ratios=[1, 2])
    
    # HYDROGRAM CHART
    ax = plt.subplot(gs[1])
    ax.plot(t,sQ)
    ax.set_ylabel(u'Q(m³/s)', color='b')
    ax.set_xlabel('Time (min.)')
    ax.tick_params(axis='y', colors='b')
    ax.xaxis.grid(b=True, which='major', color='.7', linestyle='-')
    ax.yaxis.grid(b=True, which='major', color='.7', linestyle='-')
    ax.set_xlim(min(t), max(t))
    ax.set_ylim(0, max(sQ)*1.2)

    # PRECIPITATION/HYETOGRAPH CHART
    ax2 = plt.subplot(gs[0])
    ax2.bar(t, sP, 1, color='#b0c4de')
    ax2.xaxis.grid(b=True, which='major', color='.7', linestyle='-')
    ax2.yaxis.grid(b=True, which='major', color='0.7', linestyle='-')
    ax2.set_ylabel('P(mm)')
    ax2.set_xlim(min(t), max(t))
    plt.setp(ax2.get_xticklabels(), visible=False)
    
    plt.tight_layout()
    ax2.invert_yaxis()
    plt.gcf().subplots_adjust(bottom=0.15)
    plt.show()
    #plt.savefig(filename,format='pdf')
    plt.close(fig)

Sunday, September 17, 2017

Multiple linear regression in Python

Sometimes we need to do a linear regression, and we know most used spreadsheet software does not do it well nor easily.

In the other hand, a multiple regression in Python, using the scikit-learn library - sklearn - it is rather simple.


import matplotlib.pyplot as plt
import pandas as pd
from sklearn.linear_model import LinearRegression

# Importing the dataset
dataset = pd.read_csv('data.csv')
# separate last column of dataset as dependent variable - y
X = dataset.iloc[:, :-1].values
y = dataset.iloc[:, -1].values

# build the regressor and print summary results
regressor = LinearRegression()
regressor.fit(X,y)
print('Coefficients:\t','\t'.join([str(c) for c in regressor.coef_]))
print('R2 =\t',regressor.score(X,y, sample_weight=None))

#plot the results if you like
y_pred = regressor.predict(X)
plt.scatter(y_pred,y)
plt.plot([min(y_pred),max(y_pred)],[min(y_pred),max(y_pred)])
plt.legend()
plt.show()

Thursday, August 31, 2017

Calculate the centroid of a polygon with python


In this post I will show a way to calculate the centroid of a non-self-intersecting closed polygon.

I used the following formulas,

C_{\mathrm {x} }={\frac {1}{6A}}\sum _{i=0}^{n-1}(x_{i}+x_{i+1})(x_{i}\ y_{i+1}-x_{i+1}\ y_{i})

C_{\mathrm {y} }={\frac {1}{6A}}\sum _{i=0}^{n-1}(y_{i}+y_{i+1})(x_{i}\ y_{i+1}-x_{i+1}\ y_{i})

A={\frac {1}{2}}\sum _{i=0}^{n-1}(x_{i}\ y_{i+1}-x_{i+1}\ y_{i})\;

as shown in https://en.wikipedia.org/wiki/Centroid#Centroid_of_a_polygon .

In the following code, the function receives the coordinates as a list of lists or as a list of tuples.

from math import sqrt

def centroid(lstP):
    sumCx = 0
    sumCy = 0
    sumAc= 0
    for i in range(len(lstP)-1):
        cX = (lstP[i][0]+lstP[i+1][0])*(lstP[i][0]*lstP[i+1][1]-lstP[i+1][0]*lstP[i][1])
        cY = (lstP[i][1]+lstP[i+1][1])*(lstP[i][0]*lstP[i+1][1]-lstP[i+1][0]*lstP[i][1])
        pA = (lstP[i][0]*lstP[i+1][1])-(lstP[i+1][0]*lstP[i][1])
        sumCx+=cX
        sumCy+=cY
        sumAc+=pA
        print cX,cY,pA
    ar = sumAc/2.0
    print ar
    centr = ((1.0/(6.0*ar))*sumCx,(1.0/(6.0*ar))*sumCy)
    return centr

Sunday, July 30, 2017

A Library to connect Excel with Python - PYTHON FOR EXCEL

PYTHON FOR EXCEL. FREE & OPEN SOURCE.

xlwings - Make Excel Fly!

xlwings is a BSD-licensed Python library that makes it easy to call Python from Excel and vice versa:
Scripting: Automate/interact with Excel from Python using a syntax close to VBA.

- Macros: Replace VBA macros with clean and powerful Python code.
- UDFs: Write User Defined Functions (UDFs) in Python (Windows only).

Numpy arrays and Pandas Series/DataFrames are fully supported. xlwings-powered workbooks are easy to distribute and work on Windows and Mac.

Install with: pip install xlwings;

It is already included in Winpython and in Anaconda.

Links