Text string to Excel

Sometimes we just want to handle some data, from or to excel.

Here is a tip that can help in your scripts, when you want to copy data from or to an Excel spreadsheet.

Microsoft Excel recognizes the tabulation character ('\t') as cell separation, and a line break character ('\n') as a normal line break.

For example, if you copy some string data to the clipboard, as explained in previous post, you can paste it to excel cells, if the data is separated by '\t' character.

Probability: How to do combinations without repetition

Command:
itertools.combinations(iterable, r)

Return r length subsequences of elements from the input iterable.

Combinations are emitted in lexicographic sort order. So, if the input iterable is sorted, the combination tuples will be produced in sorted order.

Elements are treated as unique based on their position, not on their value. So if the input elements are unique, there will be no repeat values in each combination.

Example:

from itertools import combinations

print list(combinations([1,2,3,4], 3))

itertools.combinations returns an iterator. An iterator is something that you can for on. So, that is why it is transformed into a list in the example.

Excelent tips for Beginners

Excelent tips !!!

https://www.codementor.io/python/tutorial/python-tricks-for-beginners

SciPy minimize example - Fitting IDF Curves

SciPy (pronounced “Sigh Pie”) is an open source Python library used by scientists, analysts, and engineers doing scientific computing and technical computing.

SciPy contains modules for optimization, linear algebra, integration, interpolation, special functions, FFT, signal and image processing, ODE solvers and other tasks common in science and engineering.

In this post I will show how to use a powerful function of SciPy - minimize.

Minimize has some methods of minimizing functions. Its official documentation is shown here.

The example used is to find coefficients of a standard rainfall IDF (intensity-duration-frequency) equation. The format of the equation is as following:

We have to input the intensity data to be fitted, and the equation as a function.

import numpy as np
from scipy.optimize import minimize

# This is the IDF function, for return periods of 10 and 25 years
def func2(par, res):
    f10 =  (par[0] * 10  **par[1])/((res[0,:]+par[2])**par[3])
    f25 =  (par[0] * 25  **par[1])/((res[0,:]+par[2])**par[3])
    erroTotQ = np.sum((f10-res[1,:])**2+(f25-res[2,:])**2)
    return erroTotQ

#durations array - in minutes
d=[5,10,15,20,25,30,35,40,60,120,180,360,540,720,900,1260,1440]

# Rainfall intensities, mm/h, same lenght as minutes
r10=[236.1,174.0,145.6,128.3,116.3,107.3,100.3,94.5,79.1,48.4,34.2,18.9,13.4,10.5,8.7,6.5,5.8]
r25=[294.5,217.1,181.6,160.0,145.1,133.9,125.1,118.0,98.7,60.4,42.7,23.6,16.7,13.1,10.8,8.1,7.2]

# the following line only gather the duration and the intensities as a numpy array,
# in order to pass all of the idf constants as one single parameter, named "valid"
valid = np.vstack((d, r10, r25))

#initial guess
param1 = [5000, 0.1, 10, 0.9]

res2 = minimize(func2, param1, args=(valid,), method='Nelder-Mead')
np.set_printoptions(formatter={'float': '{: 0.3f}'.format})
print res2

Plotting Charts with PyQGIS

In this post, I will show an example of how to plot a simple line graph in PyQGIS, with matplotlib library. This library is already included in PyQGIS.

The example code is below.

import matplotlib.pyplot as plt



# x and y data as same length lists

radius = [1.0, 2.0, 3.0, 4.0, 5.0, 6.0]

area = [3.14159, 12.56636, 28.27431, 50.26544, 78.53975, 113.09724]



plt.plot(radius, area)

plt.xlabel('Radius')

plt.ylabel('Area')

plt.title('Area of a Circle')

# show grid lines

ax = plt.axes()

ax.grid(True)

# show plot window

plt.show()

Asking for User Input

In QGIS python (PyQGIS), the python comman raw_input() does not work as expected. And it will be not very practical to enter any input in Python Console.

One good alternative is to use "Qdialog". It is a module of PyQt (PyQt4.QtGui) library.
It can show a window asking for input, or asking to choose a file or folder.

We have to import it:

from PyQt4.QtGui import QFileDialog, QInputDialog, QMessageBox

QfileDialog - Window to select files or folders. It can be getOpenFileName - for opening files, getSaveFileName, for saving files, getExistingDirectory to select a folder. Example code for these types is shown below.

fileOpen = QFileDialog.getOpenFileName(None,'Select File:', 'c:/' , '.xls (*.xls)')

saveFileName = QtGui.QFileDialog.getSaveFileName(self, 'Dialog Title', '/path/to/default/directory', selectedFilter='*.txt')

folderSel = QFileDialog.getExistingDirectory(None,"Select Folder: ", 'c:/')

QInputDialog - customizable window. An example is shown below, showing a function that calls a customizable input window with defined width and height:

def windowSize(title, question, widthW, heightW):
    winD = QInputDialog(iface.mainWindow())
    winD.setInputMode(QInputDialog.TextInput)
    winD.setWindowTitle(title);
    winD.setLabelText(question)
    winD.resize(widthW,heightW)
    ok = winD.exec_()
    textInput = winD.textValue()
    return textInput

QMessageBox - It shows only a MessageBox With some information you want. Example:

QMessageBox.information(iface.mainWindow(), "Title", "Message")

Using the Windows Clipboard

The Windows Clipboard may be very useful for transferring information for processing in quick python scripts.

To this purpose, we are going to use the "win32clipboard" library, included in python distribution.

First, we have to call the library, as usual:

import win32clipboard

Then, in the code, we have to open the Clipboard:

win32clipboard.OpenClipboard()

Once opened, we can retrieve information that is in the Clipboard, copy information to it, or clear all contentents.

To get information in Clipboard, and store its information in a variable:

clipbVar = win32clipboard.GetClipboardData()

To set it to any value:

win32clipboard.SetClipboardText('testing 123')

To clear the Clipboard:

win32clipboard.EmptyClipboard()

After using the Clipboard, we have to close it, as following:

win32clipboard.CloseClipboard()

Simple tutorial to Start with PyQt4

Good and simple.

http://zetcode.com/gui/pyqt4/

Reading Text files in Python

This is a basic funcionality, but very useful for many applications.

It opens a pure text file. You do not have to import any extra library.

The main sintax for opening files is:

file_object = open(file_name [, access_mode][, buffering])

where:
file_object -> variable
file_name -> the file you want to open
access_mode -> 'r' for read, 'rw' for read and wite text files
buffering -> size of data you read from file. If you leave this blank, it uses the system default - recommended.

To read only text files, it becomes:

file = open('yourfile.txt', 'r')

'yourfile.txt' must contain the correct folder address to it.
'r' stands for read only.

If you want all the contents of your file in one string, you can call read() method:

print file.read()

A good way to deal with text files is to turn the text into a list, each line as an list element. To do this, use the readlines() method:

lstLines = file.readlines()

After all the readings and processings you want to do with the file, you have to close it as following:

file.close()

How to use active layer in QGIS python - PyQGIS

Just use the activeLayer() as in the following code.

mylayer = qgis.utils.iface.activeLayer()
print mylayer.name(), " - ",mylayer

Step by Step - Making QGIS Python Plugins

Plugin QGIS

1. Download Plugin Builder

2. Fill in all fields. Pay attention to the tips shown when you put the mouse pointer over the field.


3. After finishing, choose the folder where the plugin folder will be created.

4. Open QTDESIGNER. It is included with QGIS. Filename is "designer.exe";

5. Make the Graphical Layout of your plugin, and save it;

6. In QGIS: Download the plugin named "PLUGIN RELOAD" - as it is experimental, you have to set your QGIS to list experimental plugins;

7. Edit your source code to do whatever you want - edit the code named “YOURPLUGINNAME”.py

8. After you finish any modifications in your graphical layout or source code, save it, and then execute the PLUGIN RELOAD plugin.

9. Your Plugin is ready.

SWMM5 in Python !

Do you use SWMM5?

Have you already wanted some nicer output?
Or some kind of presentation of results that the programs does not fit you?

So, you might be interested in the SWMM5 package for python

https://pypi.python.org/pypi/SWMM5

It gathers all information from .INP files, letting you to play with all the components and the results.

How to install:

If you have already installed pip , just open OSGEO4W and type:

pip install SWMM5

If QGIS is opened, you will have to restart it.

Documentation and examples are included in the website given.

Installing pip in PyQGIS

"pip" is a package management system used to install and manage software packages written in Python. Many packages can be found in the Python Package Index (PyPI).

Many modules are installed through "pip", so as the SWMM5 module.

Installing pip in QGIS - python - PyQgis - OSGeo4w

1 - Download pip - https://bootstrap.pypa.io/get-pip.py ;
2 - Open OSGeo4W;
3 - go to the get-pip.py folder and type:
python get-pip.py

How to Install New Python Modules / Packages in QGIS - method #1

Steps:


1 - copy source folder to C:\Program Files\QGIS Wien\apps\Python27\Lib\site-packages

2 - Open OSGeo4W Shell; go to unpacked folder, where setup.py lies - (>> cd "C:\Program Files\QGIS Wien\apps\Python27\Lib\site-packages")

3 - Type: >> python setup.py install



-----


If it shows errors regarding setuptools: you will have to install setuptools.


https://pypi.python.org/packages/source/s/setuptools/setuptools-18.4.zip#md5=38d5cd321ca9de2cdb1dafcac4cb7007


Download setuptools package, then repeat the same steps for installation.

Ps: in my opinion, this is the worst method to install modules. Next post I will tell how to use "pip" with QGIS Python.

"Hello World" in QGIS python

Steps:

1 - open code editor as described in last post
2 - copy following code in python editor window:

print 'HELLO WORLD!'

3 - press the save script button - and save it in a proper folder;

4 - press the 'Run Script' Button;

5 - See the result in the Console.

Please leave comments and opinions.

How to Learn Python

That is how I learned Python:

https://www.codecademy.com/learn/python

Excellent site. Recommended.

Opening Python Editor in QGIS

How to Use Python

In this post I will try to explain how I started using PYTHON to solve everyday work needs.

You can go to Python Official site - https://www.python.org/ - and dowload the latest packages for you operating system:
https://www.python.org/downloads/

----

However, some programs already come with a python console included - that is the case of QGIS (Quantum GIS). That is how I started to code in python - with QGIS -  http://www.qgis.org/en/site/

QGIS is a Open-Source desktop GIS - it is a computer software which is really very useful for hydrologists and hydraulic engineers. And it is continuosly evolving, getting better in every new version. At the time of thise post, the current version is the 2.12 - Lyon.

QGIS has depository of plugins - users make their own plugins and share with the community. And these plugins are all python coded.

QGIS comes with a python console. And with a code editor. And there is where the show begins!!

Upcoming posts will show reference material, screenshots, and a lot of examples.

Thanks for visiting.

About the blog

In this blog I will post all of the codes for scripts or functions that I developed or found interesting, during the development of python scripts for Hydrology and Hydraulics.