H2O Member | Wed Jul 01, 2020 | 11:08 AM | Likes: 3 

Mapping IP addresses to Geographic Locations in Python

Categories: Tech | Pubby Cash Received:  10

An IP address - a list of four segments of 3 digits numbers between 0 and 255 and connected by dots can contain more information than you think. Underneath this article, you can see where this article is posted from. This gives you an example that an IP address can be transformed to the geographical location. I'm introducing one way to realize it in Python - use API from ipinfo.io. Their free version can give the geographic location, ISP provider's name, and even GPS coordinates of the location, which should be enough for most website owners. Their API include a token, which is good for querying 50,000 times a month. The following code can be used as an example. Remember to replace the ip and token with your ip and token. import requests, json link = f'https://ipinfo.io/{ip}?token={token}' r = requests.get(link) content = json.loads(r.content) print(content) ...  Read more

H2O Member | Sun Jun 28, 2020 | 11:33 PM | Likes: 3 

Methods of Converting a String of Numbers Separated by Commas to a List of Numbers in Python

Categories: Tech | Pubby Cash Received:  20

When coding in Python, free conversion of different data types is a skill that one must master. Today, let's talk about one scenario: here is a a string of numbers separated by commas: numb_str = '1,2,3,4,5' You need to convert it to a list like this: [1,2,3,4,5]. How would you do this? There are multiple ways: Method 1: numb_str = '1,2,3,4,5' numb_str_list = list(numb_str) # Destruct all elements print(numb_str_list) # Result: ['1', ',', '2', ',', '3', ',', '4', ',', '5'] numb_list = [] # Build an empty list for item in numb_str_list: # loop over the list     if item.isdigit(): # check if an item is a number         numb_list.append(int(item)) # if yes, convert it to an integer and then append to the list print(numb_list) # Result: [1, 2, 3, 4, 5] Method 2: numb_str = '1,2,3,4,5' numb_str_list = numb_str.split(',') # Extract the element from the string except the comma print(numb_str_list) # Result: ['1', '2', '3', '4', '5'] numb_list = [] # Build an empty list for item in numb_str_list: # loop over the list     numb_list.append(int(item)) # convert it to an integer and then append to the list print(numb_list) # Result: [1, 2, 3, 4, 5] Method 3: numb_str = '1,2,3,4,5' numb_str_list = numb_str.split(',') # Extract the element from the string except the comma print(numb_str_list) # Result: ['1', '2', '3', '4', '5'] numb_list = [int(item) for item in numb_str_list] # A concise way of doing method 2 print(numb_list) # Result: [1, 2, 3, 4, 5] Of the three methods, which one do you like best? ...  Read more

H2O Member | Sun Jun 28, 2020 | 12:00 AM | Likes: 3 

Machine Learning at the Intersection of Artificial Intelligence and Data Mining

Categories: Tech | Pubby Cash Received:  10

Artificial Intelligence (AI), Machine Learning (ML), and Data Mining (DM) are very fancy words nowadays in technology fields. But do you know the differences between these terms? They are all dedicated to the transformation of data to usable knowledge and support decisions. In other words, almost all disciplines will need them. AI is a broad term for using either data or knowledge to offer solutions to existing problems that require some search or reasoning. ML is a specific subarea of AI that offers procedures learning from data. DM, is the an interdisciplinary field to discover patterns in large data sets and subsumes both ML, statistical modeling, or visualization disciplines. The following figure gives you an idea about the relationship of the three terminologies, and the bracketed numbers in the figure denote the number of papers dealing with these technologies in the water and wastewater sectors, respectively, according to Hadjimichael et al. in a paper published in 2019. With regard to the specific methods within the water and wastewater fields, artificial neural network is the most popular method followed by clustering. There are huge potentials for further exploration, as the encapsulation of ML into functional decision support system is not fully investigated. This is because: (1) there is a lack of an association between the fields of water engineering and computer engineering; (2) most ML techniques were adopted by academics with limited practical experience; (3) ML techniques are inherently multi-disciplinary and combined with problems of great complexity facing water and wastewater systems; and (4) there is a lack of user-friendly interfaces and support after product delivery. With that said, I'm working on an integration of machine learning into my discipline of water and wastewater treatment. Sound promising?...  Read more

H2O Member | Fri Jun 26, 2020 | 12:26 AM | Likes: 3 

Deployment of Writing Static html Pages to the Web Server

Categories: Tech | Pubby Cash Received:  10

In the previous post titled "How to Make Static .html Pages Accessible with Flask in Python", we've solved the obstacle of html accessibility. Now the question becomes: how to make those static html pages? The basic idea is to query the database to generate these html pages and then save these html pages to the static "html" folder. To do this, we will need to use the BeautifulSoup module I've introduced in the an earlier post. First, we try to request the pages from the web server one at a time based on the article id of each page. Then we will parse the response from the server with the BeautifulSoup module. Since the result is the BeautifulSoup class, we will need to convert it to a string, and then write the string into a html file. The tricky parts are: (1) how to write to a file if the file does not exit? (2) how to avoid the character code error (e.g. a gbk code error might be thrown out) during the writing? and (3) how to make the process efficient by not re-writing the files again and again? For the first part, I could use with open('file_name', 'w') as file:     file.write() For the second part, however, the above code will not work. To solve the character code error, I will need to import the io module, and re-write the above code like this: with io.open('file_name', 'w', encoding='utf-8') as file:     file.write() For the third part, I let a script auto-run in the background with a nested loop. In the inner loop, I query the most recent posts published during the week and only write those new files. In the outer loop, such queries and writings are controlled to be run once a week. Want to see a showcase? Access my last post via this static html page! http://gopubby.com/html/173.html...  Read more

H2O Member | Thu Jun 25, 2020 | 05:53 PM | Likes: 3 

How to Make Static .html Pages Accessible with Flask in Python

Categories: Tech | Pubby Cash Received:  10

The following code can easily start a development web server on your machine with the flask module: from flask import Flask app = Flask(__name__) if __name__ == '__main__':     app.run(debug=True) Now, you would like to make any .html static files accessible using URL routing. What should you do? Here are the steps you need to follow:
Step 1: The above code needs to be revised like this: from flask import Flask, render_template app = Flask(__name__) @app.route('/html/<string:page_name>/') def render_static(page_name):     return render_template('html/%s.html' % page_name) if __name__ == '__main__':     app.run(debug=True) Step 2: Now Flask will look for any .html files in the "templates/html" folder. Go ahead to create a folder named templates in the root, and then create a subfolder named "html" within the "templates" folder. Step 3. Transfer all your static .html files into this html folder. And then all these files are accessible by the URL http://xxx.com/html/file_name Note: You cannot directly create an html folder in the root. Otherwise, the web server cannot be started, but throw out an error "ModuleNotFoundError: No module named 'htmlentitydefs'". If you delete the folder named html in the root, the error will be gone. This is because the folder name tricked Flask. By default, Flask will look into the "templates" folder for html pages. If you try to place your .html pages into a folder named other than "templates", it won't work either. Make sure be aware of that! If you are still not satisfied with this method, because with this method, it still cannot show a URL ending with a physical .html page. As you can see in the link above, the .html file name extension is cut off in the URL. To save the .html file extension in the URL the "send_from_directory" module will do the trick. from flask import Flask, render_template, send_from_directory app = Flask(__name__, static_url_path='') @app.route('/html/<path:path>') def send_html(path):     return send_from_directory('html', path) if __name__ == '__main__':     app.run(debug=True) Create a folder named 'html' in the root, and place all static .html pages into that folder. Those files will be accessible and the full name with .html extension will also be displayed in the URL like this: http://xxx.com/html/file_name.html Now you can name a folder "html" in the root in this case. That's why I like the second solution better. ...  Read more

1 ... 3 4 5 ... 9