I am currently in need of a BackGroundSubtractor model for my project. I found that the BackGroundSubtractorMOG in OpenCV source has more parameters that can be controlled than what its Python API allows. Why is that? Is it possible to provide control over all the parameters from Python itself?
vendredi 8 mai 2015
python bind code not working as expected
Why does the following code not produce the expected result?:
from Tkinter import *
root = Tk()
def key(event):
frame.focus_set()
print "pressed", repr(event.char
frame = Frame(root, width=100, height=100)
frame.bind("<Key>", key)
frame.pack()
root.mainloop()
Database created empty when models are in a separate file?
I am reading a book about flask development, and I'm pretty new to python generally.
File layout:
Project
|
|-- App
| |-- __init__.py
| |-- models.py
|
| main.py
Code inside __init__.py:
from flask import Flask
from flask.ext.bootstrap import Bootstrap
from flask.ext.sqlalchemy import SQLAlchemy
import os
from config import options
basedir = os.path.abspath(os.path.dirname(__file__))
bootstrap = Bootstrap()
db = SQLAlchemy()
def create_app():
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] =\
'sqlite:///' + os.path.join(basedir, 'database_test.sqlite')
bootstrap.init_app(app)
db.init_app(app)
with app.app_context():
db.create_all()
print("DB created")
return(app)
I have been researching other people's issues on the site, mostly here where I found to use the with app.app_context(), and to instantiate the db without using app as part of the constructor.
Here is my models.py file:
from . import db
class User(db.Model):
__tablename__ = 'Users'
account_id = db.Column(db.Integer, primary_key=True)
personaname = db.Column(db.String(50), unique=False)
steamid = db.Column(db.String(64))
avatar = db.Column(db.String(200))
profileurl = db.Column(db.String(128))
is_personaname_real = db.Column(db.Integer)
def __repr__(self):
return '<User {0}>'.format(self.personaname)
I then rune the code from main.py which is just:
from app import create_app
app = create_app()
If I move the User class into the __init__.py function, everything is created fine. However, when the User class is inside it's own file, the database is created empty. I have tried using other ways of importing, maybe something like From app.__init__.py Import db, but that didn't work either.
Select whitespace from first line only using regex in python
I would like to match and substitute whitespace that appears in the first line of a CSV.
For example I want to substitute whitespace from the first line only with '_':
"product id","Region","Region Code"
"888","North America","GEO123"
To give:
"product_id","Region","Region_Code"
"888","North America","GEO123"
This is my current approach:
f1 = open('file1', 'r')
f2 = open('newfile', 'w')
for line in f1:
f2.write(re.sub('([\s])+', '_', line))
f1.close()
f2.close()
Which replaces all whitespace throughout the document. How can I adapt this so that it only works on the first line of text?
Filtering out redundant repeated data from a pandas dataframe
I have a dataframe that looks a like this;
index, othercols, FPN
ts1, otherStuff, val1
ts2, otherStuff, val2
ts3, otherStuff, val3
ts4, otherStuff, val4
....
tsn, otherStuff, valn
Because of the external data source a lot of these values will be repeated- so in million row dataframe there will be several pieces up to 10,000s entries long all just repeating the same data for different timestamps. For my purposes at least this repetition is not necessary, so I want to remove all the repeated rows apart from the beginning and end of each section like so;
1, 0
2, 0
3, 0
4, 5
5, 0
6, 0
becomes
1, 0
3, 0
4, 5
5, 0
6, 0
I have managed to do this, but it's slower than I would like (takes 2 mins for a single 60mb file; mostly in the apply part as seen below) and I'm thinking that there must be a better way of doing it
Here's my cobbled together solution, is there a neater/faster way of doing this?;
data=df['FPN']
shft_up=(copy.deepcopy(data)).tolist()
shft_dn=(copy.deepcopy(data)).tolist()
del shft_up[0]
shft_up=shft_up+[None]
del shft_dn[-1]
shft_dn=[None]+shft_dn
df['shft_up']=shft_up
df['shft_dn']=shft_dn
def is_rep(row):
if row['shft_dn']==row['FPN'] and row['shft_up']==row['FPN']:
return 1
else:
return 0
df['mask_col']=df.apply(lambda row:is_rep(row),axis=1,reduce=False)
df=(df[df['mask_col']==0]).drop(['shft_up','shft_dn','mask_col'],axis=1)
Text mining and NLP: from R to Python
First of all, saying that I am new to python. At the moment, I am "translating" a lot of R code into python and learning along the way. This question relates to this one replicating R in Python (in there they actually suggest to wrap it up using rpy2, which I would like to avoid for learning purposes).
In my case, rather than exactly replicating R in python, I would actually like to learn a "pythonian" way of doing what I am describing here:
I have a long vector (40000 elements) in which each element is a piece of text, for example:
> descr
[1] "dress Silver Grey Printed Jersey Dress 100% cotton"
[2] "dress Printed Silk Dress 100% Silk Effortless style."
[3] "dress Rust Belted Kimono Dress 100% Silk relaxed silhouette, mini length"
I then preprocess it as, for example:
# customized function to remove patterns in strings. used later within tm_map
rmRepeatPatterns <- function(str) gsub('\\b(\\S+?)\\1\\S*\\b', '', str,
perl = TRUE)
# process the corpus
pCorp <- Corpus(VectorSource(descr))
pCorp <- tm_map(pCorp, content_transformer(tolower))
pCorp <- tm_map(pCorp, rmRepeatPatterns)
pCorp <- tm_map(pCorp, removeStopWords)
pCorp <- tm_map(pCorp, removePunctuation)
pCorp <- tm_map(pCorp, removeNumbers)
pCorp <- tm_map(pCorp, stripWhitespace)
pCorp <- tm_map(pCorp, PlainTextDocument)
# create a term document matrix (control functions can also be passed here) and a table: word - freq
Tdm1 <- TermDocumentMatrix(pCorp)
freq1 <- rowSums(as.matrix(Tdm1))
dt <- data.table(terms=names(freq1), freq=freq1)
# and perhaps even calculate a distance matrix (transpose because Dist operates on a row basis)
D <- Dist(t(as.matrix(Tdm1)))
Overall, I would like to know an adequate way of doing this in python, mainly the text processing.
For example, I could remove stopwords and numbers as they describe here get rid of StopWords and Numbers (although seems a lot of work for such a simple task). But all the options I see imply processing the text itself rather than mapping the whole corpus. In other words, they imply "looping" through the descr vector.
Anyway, any help would be really appreciated. Also, I have a bunch of customised functions like rmRepeatPatterns, so learning how to map these would be extremely useful.
thanks in advance for your time.
Smartcard PKCS11 AES Key Gen Failure
I am attempting to create an AES 256 key on an ACOS5-64 smartcard and OMNIKEY 3121 card reader, using PKCS11 in python (using the PyKCS11 library). So far, all the "standard" operations seem to work with regards to asymmetric crypto. I have run plenty of code samples and pkcs11-tool commands, to initialize the token, set/change PINs, create RSA keypairs, etc. So, the drivers are all functional (pcscd, CCID, PKCS11 middleware).
The following code is causing a problem:
from PyKCS11 import *
import getpass
libacospkcs = '/usr/lib/libacospkcs11.so'
def createTokenAES256(lbl):
pkcs11 = PyKCS11Lib()
pkcs11.load(libacospkcs)
theOnlySlot = pkcs11.getSlotList()[0]
session = pkcs11.openSession(theOnlySlot, CKF_SERIAL_SESSION | CKF_RW_SESSION)
PIN = getpass.getpass('Enter User PIN to login:')
session.login(PIN)
t = pkcs11.getTokenInfo(theOnlySlot)
print t.label
print t.model
print t.serialNumber
template = (
(CKA_CLASS, CKO_SECRET_KEY),
(CKA_KEY_TYPE, CKK_AES),
(CKA_VALUE_LEN, 32),
(CKA_LABEL, "A"),
(CKA_PRIVATE, True),
(CKA_SENSITIVE, True),
(CKA_ENCRYPT, True),
(CKA_DECRYPT, True),
(CKA_TOKEN, True),
(CKA_WRAP, True),
(CKA_UNWRAP, True),
(CKA_EXTRACTABLE, False))
ckattr = session._template2ckattrlist(template)
m = LowLevel.CK_MECHANISM()
m.mechanism = LowLevel.CKM_AES_KEY_GEN
key = LowLevel.CK_OBJECT_HANDLE()
returnValue = pkcs11.lib.C_GenerateKey( session.session, m, ckattr, key)
if returnValue != CKR_OK:
raise PyKCS11Error(returnValue)
# Now run the method to create the key
createTokenAES256('TestAESKey')
However, I get an error when running it:
~/projects/smartcard $ python testpkcs11again.py
Enter User PIN to login:
Token #A
ACOS5-64
30A740C8704A
Traceback (most recent call last):
File "testcreateaes.py", line 43, in <module>
createTokenAES256('TestAESKey')
File "testcreateaes.py", line 40, in createTokenAES256
raise PyKCS11Error(returnValue)
PyKCS11.PyKCS11Error: CKR_ATTRIBUTE_VALUE_INVALID (0x00000013)
The thing is that if I switch the CKA_TOKEN line to False, then it "works". Of course, by setting that to false, it makes the key a session object instead of a token object (i.e. after I logout, the key is wiped). Using pkcs11-tool with --list-objects, the key is not there. I can use the ACSCMU (GUI tool for token admin), I can create an AES key in the "Secret Key Manager" and it does create a persistent key. But I have no way to see what the ACSCMU is doing to make it persistent (it may not be using PKCS11 at all).
If I had to guess the problem, I'd guess that it has to do with the session. If CKA_TOKEN=True is invalid, then it seems the token is not actually in RW mode (as suggested by the CKF_RW_SESSION in the 9th line). So far, I'm not sure what else to try or how to debug this.
Django Password Reset shows IP instead of domain name
I am trying to use Django's generic view for resetting password.
from django.contrib.auth.views import password_reset
This is the email template I am using with it:
<p>{% trans "Please go to the following page and choose a new password:" %}<br>
{% block reset_link %}
{{ protocol }}://{{ domain }}{% url 'password_reset_confirm' uidb64=uid token=token %}
{% endblock %}</p>
Here the {{ domain }} part is not returning my domain name but rather my IP address like http://127.0.0.1:8000/
The subject line also says something like this: Password reset on 127.0.0.1:8000
How do I get the domain name instead of the IP address?
I tried adding
SITE_URL = 'mydomain.com' and SITE_NAME = 'My Website'
to settings.py, but it didn't work.
Thanks in advance.
'/' in names in HDF5 files confusion
I am experiencing some really weird interactions between h5py, PyTables (via Pandas), and C++ generated HDF5 files. It seems that, h5check and h5py seem to cope with type names containing '/' but pandas/PyTables cannot. Clearly, there is a gap in my understanding, so:
What have I not understood here?
The gory details
I have the following data in a HDF5 file:
[...]
DATASET "log" {
DATATYPE H5T_COMPOUND {
H5T_COMPOUND {
H5T_STD_U32LE "sec";
H5T_STD_U32LE "usec";
} "time";
H5T_IEEE_F32LE "CIF/align/aft_port_end/extend_pressure";
[...]
This was created via the C++ API. The h5check utility says the file is valid.
Note that CIF/align/aft_port_end/extend_pressure is not meant as a path to a group/node/leaf. It is a label, that we use internally which happens to have some internal structure that contains '/' as delimiters. We do not want the HDF5 file to know anything about that: it should not care. Clearly, if '/' are illegal in any HDF5 name, then we have to change that delimiter to something else.
Using PyTables (okay, Pandas but it uses PyTables internally) to read the file, I get an
>>> import pandas as pd
>>> store = pd.HDFStore('data/XXX-20150423-071618.h5')
>>> store
/home/XXX/virt/env/develop/lib/python2.7/site-packages/tables/group. py:1156: UserWarning: problems loading leaf ``/log``::
the ``/`` character is not allowed in object names: 'XXX/align/aft_port_end/extend_pressure'
The leaf will become an ``UnImplemented`` node.
I asked about this in this question and got told that '/' are illegal in the specification. However, things get stranger with h5py...
Using h5py to read the file, I get what I want:
>>> f['/log'].dtype
>>> dtype([('time', [('sec', '<u4'), ('usec', '<u4')]), ('CI
F/align/aft_port_end/extend_pressure', '<f4')[...]
Which is more or less what I set out with.
Needless to say, I am confused. Have I managed to create an illegal HDF5 file that somehow passes h5check? Is PyTables not supporting this edge case? ... I am confused.
Clearly, I could write a simple wrapper something like this:
>>> import matplotlib.pyplot as plt
>>> silly = pd.DataFrame(f['/log']['CIF/align/aft_port_end/extend_pressure'])
>>> silly.plot()
>>> plt.show()
to get all the data from the HDF5 file into Pandas. However, I am not sure if this is a good idea because of the confusion earlier. My biggest worry is the conversion might not scale if the data is very large...
Whether to (and how to) handle multiple many-to-many relationship through a common intermediary/join table in django?
I have several models that implement a so called 'Taggable' behavior:
class Tag(models.Model):
name = models.CharField(max_length=200)
class Taggable(models.Model):
tags = models.ManyToManyField(Tag)
class Meta:
abstract = True
class A(Taggable):
...
class B(Taggable):
...
class C(Taggable):
...
class D(Taggable):
...
This scenario causes an intermediary table to be created for each and every model inheriting from Taggable. I.e.,
appname_a_tags
appname_b_tags
appname_c_tags
appname_d_tags
I am still at the beginning of the development, and the number of such models and thus tables might increase. So, it bothers me a little to have a clutter of tables with similar data, plus, in the future, I might need common functionality for all the tags assigned in the application (e.g., collecting statistical data of which tags used where, or maybe for a search feature).
Now, my question is: from general engineering perspective would it be feasible to use one common intermediary/join table for all of the models consuming this 'Taggable' behavior? If so, what would be the disadvantages of doing so, and would be the best approach to tackle this.
Being new in django, I would have tried (or researched more on) these 2 scenarios if I had to do it:
SCENARIO 1:
- include a 'class' field in the Tag model above that will keep the type of ojbect that the tag is assigned to (e.g., A, B, C, or D)
- create a proxy Tag class for each of A, B, C, D (e.g., ATag, BTag, ..), override the models.Model class' 'save' method in order to update 'class' column appropriately. I would also have to customize model's manager in order to filter and properly retrieve tags that belong to that class only.
-
let each class consume its custom Tag class. I.e.,
class A(models.Model): tags = models.ManyToManyField(ATags) ...
SCENARIO 2:
- create a common intermediary model TagAssigned, and include the extra 'class' field besides the 2 foreign keys for the tag and the item tagged
-
use TagAssigned as intermediary table for all 4 classes. E.g.,
Class A(models.Model) tags = models.ManyToManyField(Tag, through='TagAssigned') ... -
override and customize the 'save' methods and the model managers of classes A, B, C, and D (as opposed to proxy models in scenario 1)
Is a mutex really necessary in this piece of code?
The code below shows a class which I use for PyQt threading funcionality. In my main program I instantiate and start this thread (i.e. I use PyQt's moveToThread, and countdown is the run method.). While the thread is running I frequently call the reset method. I'm not sure if the mutex I've implemented is actually necessary, I hope somebody can clear this up for me. I've executed a quick test in which I've commented out the mutex, this gave me no problems, even when I comment out the time.sleep(1) and call the reset method while the thread is running without any delay. However I wan't to be 100% secure, hence the reason for my question here.
import time
from PyQt4.QtCore import *
from threading import Lock
class Countdown(QObject):
finished = pyqtSignal()
def __init__(self, countdownInSec=30):
super(Countdown, self).__init__()
self.COUNTDOWN_IN_SEC = countdownInSec
self._countdownInSec = self.COUNTDOWN_IN_SEC
self._mutex = Lock()
@pyqtSlot()
def countdown(self):
while self._countdownInSec > 0:
print(self._countdownInSec)
self._mutex.acquire()
try:
self._countdownInSec -= 1
finally:
self._mutex.release()
time.sleep(1)
self.finished.emit()
def increment(self, seconds):
self._mutex.acquire()
try:
self._countdownInSec += seconds
finally:
self._mutex.release()
def reset(self):
self._mutex.acquire()
try:
self._countdownInSec = self.COUNTDOWN_IN_SEC
finally:
self._mutex.release()
extraction of main (only the part that is relevant for this question)
fpIntervUpdCountdownReset = pyqtSignal()
def __init__(self):
self.initFlightPathIntervUpdater()
def initFlightPathIntervUpdater(self):
self.fpIntervUpdCountdownThr = QThread()
self.fpIntervUpdCountdown = countdown.Countdown()
self.fpIntervUpdCountdown.moveToThread(self.fpIntervUpdCountdownThr)
self.fpIntervUpdCountdown.finished.connect(self.fpIntervUpdCountdownThr.quit)
self.fpIntervUpdCountdown.finished.connect(self.flightPathIntervUpdate)
self.fpIntervUpdCountdownThr.started.connect(self.fpIntervUpdCountdown.countdown)
def flightPathIntervUpdateReq(self):
if self.fpIntervUpdCountdownThr.isRunning():
self.fpIntervUpdCountdown.reset()
else:
print 'start'
self.fpIntervUpdCountdownThr.start()
@pyqtSlot()
def flightPathIntervUpdate(self):
print "perform flightPathIntervUpdate"
How can I log from Python to syslog with either SysLogHandler or syslog on Mac OS X *and* Debian (7)
I've followed several answers here on SO to no avail.
I'm developing on a Macbook (Yosemite), but our test/production boxes are Debian 7 (using rsyslog). I'm trying to log out to syslog in a way that will work both locally and not.
I tried the option of using SysLogHandler. This works on Mac:
import logging
import logging.handlers
import syslog
h = logging.handlers.SysLogHandler(address='/var/run/syslog', facility=syslog.LOG_LOCAL1)
h.ident = 'works_on_macs'
logger = logging.getLogger('i_am_a_lumberjack')
logger.addHandler(h)
logger.debug("And I don't care")
logger.info('There is a sale on today')
logger.warn('Do not touch the hot stove!')
logger.error('Sorry, times up')
logger.critical('That sure is an ugly tie')
These messages will show up in my mac syslog. However, when I change address='/dev/log' on Debian 7... no dice.
Yet:
import syslog
syslog.openlog(ident='im_a_lumberjack', facility=syslog.LOG_LOCAL1)
syslog.syslog(syslog.WARNING, 'Watch out!')
Works on Debian 7, but not on Mac.
I would really love to be able to get one logging solution that works for both platforms. Obviously the address is going to be different, but I'm already setting that in config.
So how do I get syslog working both for Mac and Debian?
How can I access session object in CKAN extension?
I am writing an CKAN extension, where I am implementing CKAN interface IAuthenticator and I need to save some additional information about logged-in user in session. I have found out that CKAN uses beaker session. How can I access it? In the documentation of CKAN is just one sentence: The Session object is available through the toolkit.
Can you help me please?
FindWindowEx doesn't find MessageBox appearing over Remote Desktop Connection
We have a build machine on which we do daily builds and execute tests at the application we develop. The problem is that some tests are failing because some of our executables are crashing. And if they would crash normally it would just be a failed test.
But instead of that they fail with a popup that prevents them from finishing. They will be killed after some determined time (5-10 minutes usually). We overcome this issue by creating a "watchdog" that periodically checks for popups and closes them when found. The python code for checking is here:
def CheckGenericPopupByClassName(hwnd,className):
# pass None for desktop popups
hwndPopup = None
hwndFirst = None
consecutiveExceptionCount = 0
# check for popups on Desktop
while True:
try:
hwndPopup = win32gui.FindWindowEx(hwnd, hwndPopup, className, None) # Check with Spy++ for class name
except Exception as e:
print("CheckGenericPopupByClassName exception:"+str(e))
hwndPopup = hwndFirst = None
consecutiveExceptionCount = consecutiveExceptionCount + 1
if consecutiveExceptionCount > 5:
return
continue
consecutiveExceptionCount = 0
if hwndPopup is None or hwndPopup is 0 or hwndPopup is hwndFirst:
break
if hwndFirst is None:
hwndFirst = hwndPopup
HandleGenericPopup(hwndPopup) # this closes the popup
The problem is that the MessageBox is above the remote desktop connection login and is not found by the previous method. After I login to the remote desktop connection popups are found by the function that is periodically called.
The MessageBox is from csrss.exe (I saw this with Process Explorer) and has the following text:
"XXXXX.exe - Application Error"
"The instruction at <...> referenced memory at <...> . The memory could not be read."
Click on OK to terminate the program
Click on CANCEL to debug the program
I could do this: Can the "Application Error" dialog box be disabled?
But I want to know why FindWindowEx doesn't find the MessageBox in this case. Any ideas what should I do to find that MessageBox?
Thanks!
Later Edit: The solution to disable the popups can be found here.
Using Boost.Python build shared lib in Blender through Python
What I currently try to achieve is building a python mapping of my C++ classes through Boost.Python. After this I want to use the resulting shared library in a blender add-on to be able to take advantage of already existing functionality coming from the mapped C++ classes.
I can already build my shared library and write sample scripts in python, which are using my library as well.
Everything fine here but the problem is that as soon as I try to use it in an add-on, Blender 2.74 keeps crashing all the time as soon as I add an import statement with this little hint in the crash report:
6 libboost_python.dylib 0x000000010aa7cc3e boost::python::detail::init_module(PyModuleDef&, void (*)()) + 30 (module.cpp:44)
In module.cpp inside of boost line 41-46:
BOOST_PYTHON_DECL PyObject* init_module(PyModuleDef& moduledef, void(*init_function)())
{
return init_module_in_scope(
PyModule_Create(&moduledef),
init_function);
}
My boost 1_58 is compiled using Python 3.4.2:
otool -L /usr/local/lib/libboost_python.dylib
/usr/local/lib/libboost_python.dylib:
libboost_python.dylib (compatibility version 0.0.0, current version 0.0.0)
/Library/Frameworks/Python.framework/Versions/3.4/Python (compatibility version 3.4.0, current version 3.4.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)
Running the python3 bin from that directory gives me:
python3
Python 3.4.2 (v3.4.2:ab2c023a9432, Oct 5 2014, 20:42:22)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
The Blender Python console gives me:
PYTHON INTERACTIVE CONSOLE 3.4.2 (default, Nov 25 2014, 12:01:44) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.54)]
Command History: Up/Down Arrow
Cursor: Left/Right Home/End
Remove: Backspace/Delete
Execute: Enter
Autocomplete: Ctrl-Space
Zoom: Ctrl +/-, Ctrl-Wheel
Builtin Modules: bpy, bpy.data, bpy.ops, bpy.props, bpy.types, bpy.context, bpy.utils, bgl, blf, mathutils
Convenience Imports: from mathutils import *; from math import *
Convenience Variables: C = bpy.context, D = bpy.data
>>>
My own project uses this python version as well. Also Blender uses this Python version.
I really don't know what to try next here since in standalone mode EVERYTHING works as expected. Also the fact that the crash even occurs as soon as I run a new script inside Blenders Text Editor having the import statement.
Anybody having experience with Boost.Python and Blender?
Thanks in advance
How do you check a condition of several pandas DataFrame.Series element-wise and apply the result to a new column?
I have a pandas.Dataframe.
df = pandas.DataFrame([(11,12,13),(1,3,5),(1,1,2)], columns=['a','b','c'])
a b c
0 11 12 13
1 1 3 5
2 3 1 2
I would like to create a fourth column called d which would tell me if all elements in a row are bigger than 10. It this example it would look like this.
a b c d
0 11 12 13 True
1 1 3 5 False
2 3 1 2 False
I tried this which gives me a TypeError.
x = df['a']
y = df['b']
z = df['c']
df['d'] = df.apply(lambda x,y,z: True if x > 10 and y > 10 and z > 10 else False)
I have also tried this which gives me a ValueError.
df['d'] = True
df['e'] = df['d'].where(df['a'] > 10 and df['b'] > 10 and df['c'] > 10, other=False)
Python - stopping a while loop issue
I have a while loop in an algorithm:
if Direction == D:
while maze[x][y][0] and not maze[x][y][1]:
if (x, y) == (2,2):
return Output
y += 1
Output.append((x, y))
return Output
Which produces a list of tuples and I have to break the function with the return to do so.
However, when I attempt to turn this into some pythonic thing and break it with:
if Direction == D:
while maze[x][y][0] and not maze[x][y][1]:
if (x, y) == (2,2):
break
y += 1
Output.append((x, y))
return Output
The terminal won't even return something. Is anyone able to tell why? Am I able to just 'cancel' a loop and move on?
Kivy: How to avoid overlapping BoxLayout widgets
I have a Kivy app that is comprised of 3 rows. The first row is an ActionBar, the second row is a search field and button, and the last row is some text (split into two columns). My problem is that the last row of text is "overlapping" the 1st and 2nd rows. Here is a screen grab to illustrate the problem:
I used Paint to alter the image to depict what I'm trying to achieve:
I was following along to the suggestions for using nested layouts here, but I couldn't get it to work properly. Here is my main.py:
from kivy.app import App
from kivy.uix.boxlayout import BoxLayout
class ShowActionBar(BoxLayout):
pass
class ShowApp(ShowActionBar):
pass
class MyTestApp(App):
def build(self):
return ShowApp()
if __name__ == "__main__":
MyTestApp().run()
Here is my kv file:
<ShowActionBar>:
ActionBar:
ActionView:
use_seperator: True
ActionPrevious:
title: "Magic Buffet"
with_previous: False
ActionOverflow:
ActionButton:
text: "Settings"
ActionButton:
text: "Log Out"
<ShowApp>:
orientation: "vertical"
BoxLayout:
orientation: "horizontal"
height: "35dp"
size_hint_y: None
TextInput:
height: 30
width: 200
size_hint_y: None
size_hint_x: None
focus: False
multiline: False
hint_text: "What would you like to Eat?"
Button:
height: 30
width: 100
size_hint_y: None
size_hint_x: None
text: "Search Foods"
BoxLayout:
orientation: "horizontal"
MenuLeftColumn:
MenuRightColumn:
# Blank label to fill out the remaining bottom space
Label:
<MenuLeftColumn@BoxLayout>:
orientation: "vertical"
BoxLayout:
orientation: "horizontal"
height: 30
width: 400
size_hint_y: None
size_hint_x: None
Label:
text: "Menu Item 1"
font_size: "20dp"
# The same "Menu Item 1" widget is repeated multiple
# times to demonstrate a long list of widgets.
# I've removed it for brevity
BoxLayout:
orientation: "horizontal"
height: 30
width: 400
size_hint_y: None
size_hint_x: None
Label:
text: "Menu Item 1"
font_size: "20dp"
<MenuRightColumn@BoxLayout>:
orientation: "vertical"
BoxLayout:
orientation: "horizontal"
height: 30
width: 400
size_hint_y: None
size_hint_x: None
Label:
text: "Price"
font_size: "20dp"
What is the correct way to stack widgets (nested or otherwise) so that this overlapping behavior is avoided?
Plot 2d line diagram in mayavi
I have a dataset of a tennis game. This dataset contains the ball positions in each rally and the current score. I already 3d-visualized the game and ball positions in mayavi.
Now I want to plot 2d line diagrams in mayavi that visualizes the score developement after specific events (such as after: a break, a set-win, set-loss,...).
I came up with some ideas, but none of them are satisfying:
- I could use imshow and "draw" the diagram
- I could use points3d to plot the diagram
- Maybe I can somehow use pyplot to plot the diagram, then make a screenshot und then plot this screenshot in mayavi... Any idea if this is possible?
Do you have any other idea how I could plot a 2d line diagram in mayavi?
Why isn't this sorting algorithm more widely used?
I was graphing out letter frequency in some large academic documents. As part of this process, is was sorting letters from large clippings of those documents into alphabetical order. I was using Python's built in sorted function, and I started to wonder if I could make it faster. I then wrote the following function:
def count_sort(l):
items = {'a':0,'b':0,'c':0,'d':0,'e':0,'f':0,'g':0,'h':0,'i':0,'j':0,'k':0,'l':0,'m':
0,'n':0,'o':0,'p':0,'q':0,'r':0,'s':0,'t':0,'u':0,'v':0,'w':0,'x':0,'y':0,'z'
:0}
for item in l:
items[item] += 1
sort_l = []
for key in items:
sort_l += key*items[key]
return sort_l
When testing this code versus sorted on a 10000 letter long string of text, it was almost 20X faster.
With such a performance boost, why isn't this sorting algorithm in the standard libs?
Iterate through xml and html with BeautifulSoup
Currently I'm trying to use BeautifulSoup to iterate over a xml file, trying to extract these kind of lines:
<Cell ss:StyleID="s(26|27|39)"><Data ss:Type="String">Textcontent</Data></Cell>
Extracting them works mostly, but I don't know why I'm getting more than 250 results when I should be getting only 1.
My current script:
#!/usr/local/bin/python
# coding: utf-8
import sys
from HTMLParser import HTMLParser
from bs4 import BeautifulSoup as bs
class ParseExcelXMLFile(object):
soup = None
substituteTagDict = {
"ul": "",
"b": "*",
"li": "\n * ",
}
_file = None
_index = []
def __init__(self, file):
self._file = file
self.soup = bs(open(file, 'r+').read())
self.iterateXML(self.soup)
def iterateXML(self, node):
if hasattr(node, 'name') and node.name == u'data':
self.parseHtml(node)
if 'findChildren' in dir(node):
for child in node.findChildren():
self.iterateXML(child)
def iterateHtml(self, node):
processed_html = ""
for child in node.findChildren():
if hasattr(child, 'name'):
processed_html += str(self.substituteHtmlTag(child))
if hasattr(child, 'findChildren'):
processed_html += self.iterateHtml(child)
return processed_html
def substituteHtmlTag(self, node):
if node.name in self.substituteTagDict:
node.name = self.substituteTagDict[node.name]
# index the other attributes of the tag and save it in another file, while remark to index remains like [#431]
# not sure how I can access them without knowing the identifier of them(name, id, ...), similar to:
# if not node.attributes in self._index:
# self._index.append(node.attributes)
# node.attributes = "[#%i]" % (self._index.index(node.attributes))
return node.prettify()
def parseHtml(self, node):
parsed_html = ""
html = HTMLParser.unescape.__func__(HTMLParser, node.text)
if '<ul class="bodytextAttributes">' in html:
soup = bs(html)
for bodytext in soup.find_all('ul', {'class' : 'bodytextAttributes'}):
parsed_html += self.iterateHtml(bodytext)
return parsed_html
def saveXML(self, file=_file):
open(file, 'w+').write(str(self.soup.prettify()))
if __name__ == '__main__':
parser = ParseExcelXMLFile(sys.argv[1])
Since I failed already by iterating over the xml elements the html part is written freely without testing sadly. For the test case I've rewritten the structure of the xml file:
<?xml version="1.0"?>
<Workbook xmlns="urn:schemas-microsoft-com:office:spreadsheet"
xmlns:o="urn:schemas-microsoft-com:office:office"
xmlns:x="urn:schemas-microsoft-com:office:excel"
xmlns:html="http://ift.tt/qQdaDR"
xmlns:ss="urn:schemas-microsoft-com:office:spreadsheet">
<DocumentProperties xmlns="urn:schemas-microsoft-com:office:office">
<Author>Test Case</Author>
</DocumentProperties>
<OfficeDocumentSettings xmlns="urn:schemas-microsoft-com:office:office">
<AllowPNG/>
</OfficeDocumentSettings>
<Styles>
<Style ss:ID="s27">
<Alignment ss:Vertical="Top" ss:WrapText="1"/>
</Style>
<Style ss:ID="s35">
<Font ss:FontName="Verdana" ss:Size="18.0" ss:Bold="1"/>
<Interior ss:Color="#969696" ss:Pattern="Solid"/>
</Style>
</Styles>
<Worksheet ss:Name="Translation">
<Table ss:ExpandedColumnCount="6" ss:ExpandedRowCount="5" x:FullColumns="1" x:FullRows="1">
<Column ss:AutoFitWidth="0" ss:Width="151.0"/>
<!-- Page header -->
<Row>
<Cell ss:Index="2" ss:StyleID="s35"><Data ss:Type="String">TestString [1]</Data></Cell>
<Cell ss:StyleID="s35"></Cell>
<Cell ss:StyleID="s35"></Cell>
<Cell ss:StyleID="s35"></Cell>
</Row>
<!-- Field list header -->
<Row>
<Cell ss:Index="2" ss:StyleID="s38"><Data ss:Type="String">Fieldname:</Data></Cell>
<Cell ss:StyleID="s38"><Data ss:Type="String">Source language:</Data></Cell>
<Cell ss:StyleID="s38"><Data ss:Type="String">Alternative source language:</Data></Cell>
<Cell ss:StyleID="s38"><Data ss:Type="String">Translation:</Data></Cell>
<Cell ss:StyleID="s38"><Data ss:Type="String">Difference since last tr.:</Data></Cell>
</Row>
<!-- Element header -->
<Row>
<Cell ss:Index="2" ss:StyleID="s37"><Data ss:Type="String">Element: pages:1</Data></Cell>
<Cell ss:StyleID="s37"></Cell>
<Cell ss:StyleID="s37"></Cell>
<Cell ss:StyleID="s37"></Cell>
</Row>
<!-- Translation row: -->
<Row ss:StyleID="s25">
<Cell><Data ss:Type="String">translation[pages][1][pages_language_overlay:130:title]</Data></Cell>
<Cell ss:StyleID="s26"><Data ss:Type="String">title</Data></Cell>
<Cell ss:StyleID="s27"><Data ss:Type="String"><ul class="bodytextAttributes"> <li class="field"> <b>bold:</b> <span class="exampleClass">1234567890</span> </li> <li class="field"> <b>bold:</b> <div class="exampleClass2"> <ul class="exampleClass3"> <li style="padding-left: 10px;">Text1</li> </ul> <ul class="exampleClass3"> <li style="padding-left: 10px;">Text2</li> </ul> <ul class="exampleClass3"> <li style="padding-left: 10px;">Text3</li> </ul> <ul class="exampleClass3"> <li style="padding-left: 10px;">Text4</li> </ul> </div> </li></ul></Data></Cell>
<Cell ss:StyleID="s27"><Data ss:Type="String"></Data></Cell>
<Cell ss:StyleID="s39"><Data ss:Type="String"><ul class="bodytextAttributes"> <li class="field"> <b>bold:</b> <span class="exampleClass">1234567890</span> </li> <li class="field"> <b>bold:</b> <div class="exampleClass2"> <ul class="exampleClass3"> <li style="padding-left: 10px;">Text1</li> </ul> <ul class="exampleClass3"> <li style="padding-left: 10px;">Text2</li> </ul> <ul class="exampleClass3"> <li style="padding-left: 10px;">Text3</li> </ul> <ul class="exampleClass3"> <li style="padding-left: 10px;">Text4</li> </ul> </div> </li></ul></Data></Cell>
<Cell ss:StyleID="s27"><Data ss:Type="String">[No change]</Data></Cell>
</Row>
<!-- Spacer row -->
<Row>
<Cell ss:Index="2"><Data ss:Type="String"></Data></Cell>
</Row>
</Table>
<WorksheetOptions xmlns="urn:schemas-microsoft-com:office:excel">
<PageSetup>
<Layout x:Orientation="Landscape"/>
</PageSetup>
</WorksheetOptions>
</Worksheet>
<Worksheet ss:Name="Information">
<WorksheetOptions xmlns="urn:schemas-microsoft-com:office:excel">
<PageSetup>
<Layout x:Orientation="Landscape"/>
</PageSetup>
</WorksheetOptions>
</Worksheet>
</Workbook>
any hints why I got so many results in parsing the excel xml with my script and how I could get the amount I really need? I feel like I'm getting the children twice and threefold, how can I prevent that? I should be able to index and substitute the tags, but iterating through the xml/html is really not working for me atm(with the current script/xml I get 256 results, which should be 1).
Python delete selected files from a directory [duplicate]
This question is an exact duplicate of:
I want to remove only unwanted files from directory using python. There are files like 1_x.dat, 1_y.dat, 2_x.dat, 2_y.dat,...50_x.dat, 50_y.dat present in my working directory. Now I want to delete only 1*, 2*, 3*, 6*, 8* and 30* files. I ran following script from my working directory
import os
num=[1,2,3,6,8,30]
num=" ".join(str(x)+'*' for x in num)
os.remove(num)
but it complaining
OSError: [Errno 2] No such file or directory: '1* 2* 3* 6* 8* 30*'
Your any help will be appreciated.
Thanks
-Viral
python 2.7 divide list produced from counter into separate columns in csv
I have two columns of data from a csv file that I've counted (basically the number of tags per chapter in a novel). I now need to separate this data into three columns: tag, chapter number, and count. Here's what I have so far:
with open('test.csv', 'r') as z:
reader = csv.reader(z)
for row in reader:
labels.append(row[0])
chapLabels.append(int(row[1]))
combo = zip(labels, chapLabels)
c = collections.Counter(combo)
d = c.items()
with open('output.csv', 'w') as csvfile:
writer = csv.writer(csvfile, delimiter=",")
writer.writerows(d)
This outputs two columns, the first is a list of my counts that includes a string and an integer in this format: ("tag1", 37); the second is the chapter number. I'm still a beginner but have searched widely for an answer to this question and have come up short.
Trying to input a lot of Data into mysqldb
#!/usr/bin/env python
from __future__ import print_function
import pymysql.cursors
Journal=open('J_15April.txt')
jrn=Journal.read()
Zeile=jrn.split('14.04.2015')
conn = pymysql.connect(host='localhost', port=3306, user='root',passwd='',db='daten', cursorclass=pymysql.cursors.DictCursor)
ean = int(row[0])
smr = char(row[1])
uhrzeit = char(row[2])
for line in Zeile:
cur = conn.cursor()
if 'BOTTLE_ID' in line
line2 = line.split(';')
line3 = line2[0].strip(' BOTTLE_UNIQUE: BOTTLE_ID: 0')
uhrzeit = (line3[:-4])
if 'EAN' not in line:
ean = 0
else:
ean = line2[7].strip('EAN: ')
if ' SMR: 1;' in line:
smr = ('gelesen')
else:
smr = ('nicht gelesen')
cur.execute("INSERT INTO lga (ean, smr, uhrzeit) VALUES ('%s','%s','%s')%(ean, smr, uhrzeit)")
cur.fetchall()
cur.close()
conn.commit()
conn.close()
Hi guys,
im currently trying to put a lot of data into a mysqldb. I wrote two seperate programms, one is for filtering the important stuff out of a very long .txt and the other one inserts them in the database.
The actual problem now occured when i tried combine those two programm to automatically read and insert the data. I think the structure of the shown programm my be a little chaotic and thats the main reason why it is not working. Im absolutely new to python and it was quite hard for me two make the two other programms work so please help me out.
If you are interested in the two single functions i can post them as well.
greetings
trbo
Python: datetime.date "expected datetime, got int"
So, for a long time now, I have been using code like the following in Python for version tracking:
...
import datetime
...
"""
Version tracking
"""
__version__ = 4.0
__dateV__ = datetime.date(2015, 5, 7)
...
Now, out of the blue, I get an error saying "TypeError: descriptor 'date' requires a 'datetime.datetime' object but received a 'int'"
This has never happened before, and the documentation for the datetime module says that a "date" object should be initialized with year, month, and day arguments, which are all INTEGERS.
I have confirmed that I am using Python version 2.7.
Has anyone else seen this error? This is literally at the top of my code (right after all of the imports.) Please help.
python cannot get folders to save inside of one folder
I've run into some trouble that I believe must be an easy fix. I am currently pulling png files from one path, and saving them in a folder in another path. This is all done correctly. What I try to do after is to group those folders into one Results folder. My current code does not throw any exceptions, the results folder is created, but the other folders are not moving into the Results folder.
def create_folder(self,folder_name):
'''Create folders if does not exist'''
path,fileName = os.path.split(self.snmpPaths[0])
target = os.path.join(path, folder_name)
if not os.path.exists(target):
os.makedirs(target)
return target
snippet from other method
if flag ==1:
for png in pngs:
if 'MPM' in png:
target = self.create_folder("MPM")
folderList.add(target)
elif 'DAT' in png:
target = self.create_folder("DAT")
folderList.add(target)
elif 'HNR' in png:
target = self.create_folder("HNR")
folderList.add(target)
elif 'VER' in png:
target = self.create_folder("VER")
folderList.add(target)
elif 'XT2R' in png:
target = self.create_folder("XT2R")
folderList.add(target)
else:
target = self.create_folder("Other")
folderList.add(target)
try:
print "png =", png
print "target = ", target
os.rename(png, os.path.join(target, png))
print "png after =", png
except BaseException:
print "Could not rename file for target"
for folder in folderList:
# path,folder = os.path.split(folder)
# folder = "./%s" %(folder)
try:
print "folder = ", folder
print "results = ", results
os.rename(folder, os.path.join(results, folder))
print "folder after rename = ", folder
except BaseException:
print "Could not rename file for results"
Here is a sample of my print statements. I can provide more if needed. Any help will be appreciated. I tried to copy the same format as the png files (./FILE) as seen in my commented code, but I would get a BaseException. An explanation would be great as to why my code is not performing the way I want it to. I'm guessing it has to do with my folder variable.
png = ./VER_SUBMODE.png
target = D:\SNMP\SPINACH\VER
png after = ./VER_SUBMODE.png
png = ./VER_TX_STATUS.png
target = D:\SNMP\SPINACH\VER
png after = ./VER_TX_STATUS.png
folder = D:\SNMP\SPINACH\VER
results = D:\SNMP\SPINACH\Results_2015-05-08_09-21-01
folder after rename = D:\SNMP\SPINACH\VER
Replicating request to Chef with Python RSA
Problem: I need to have a Python 3(so pychef is out of the question) wrapper for Chef's REST API.
I approached it by trying to replicate the authorisation curl script shown here: http://ift.tt/1EnQt6u using a python rsa package: http://ift.tt/1cgsbNC
Here's my rewrite(it could be simpler but I started getting paranoid about line breaks and headers order, so added a few unnecessary things):
import base64
import hashlib
import datetime
import rsa
import requests
import os
from collections import OrderedDict
body = ""
path = "/nodes"
client_name = "anton"
client_key = "/Users/velvetbaldmime/.chef/anton.pem"
# client_pub_key = "/Users/velvetbaldmime/.chef/anton.pub"
hashed_body = base64.b64encode(hashlib.sha1(body.encode()).digest()).decode("ASCII")
hashed_path = base64.b64encode(hashlib.sha1(path.encode()).digest()).decode("ASCII")
timestamp = datetime.datetime.now().strftime("%Y-%m-%dT%H:%M:%SZ")
canonical_request = 'Method:GET\\nHashed Path:{hashed_path}\\nX-Ops-Content-Hash:{hashed_body}\\nX-Ops-Timestamp:{timestamp}\\nX-Ops-UserId:{client_name}'
canonical_request = canonical_request.format(
hashed_body=hashed_body, hashed_path=hashed_path, timestamp=timestamp, client_name=client_name)
headers = "X-Ops-Timestamp:{timestamp}\nX-Ops-Userid:{client_name}\nX-Chef-Version:0.10.4\nAccept:application/json\nX-Ops-Content-Hash:{hashed_body}\nX-Ops-Sign:version=1.0"
headers = headers.format(
hashed_body=hashed_body, hashed_path=hashed_path, timestamp=timestamp, client_name=client_name)
headers = OrderedDict((a.split(":", 2)[0], a.split(":", 2)[1]) for a in headers.split("\n"))
headers["X-Ops-Timestamp"] = timestamp
with open(client_key, 'rb') as privatefile:
keydata = privatefile.read()
privkey = rsa.PrivateKey.load_pkcs1(keydata)
with open("pubkey.pem", 'rb') as pubfile:
keydata = pubfile.read()
pubkey = rsa.PublicKey.load_pkcs1_openssl_pem(keydata)
signed_request = base64.b64encode(rsa.sign(canonical_request.encode(), privkey, "SHA-1"))
dummy_sign = base64.b64encode(rsa.sign("hello".encode(), privkey, "SHA-1"))
print(dummy_sign)
def chunks(l, n):
n = max(1, n)
return [l[i:i + n] for i in range(0, len(l), n)]
auth_headers = OrderedDict(("X-Ops-Authorization-{0}".format(i+1), chunk) for i, chunk in enumerate(chunks(signed_request, 60)))
all_headers = OrderedDict(headers)
all_headers.update(auth_headers)
# print('curl '+' \\\n'.join("-H {0}: {1}".format(i[0], i[1]) for i in all_headers.items())+" \\\nhttps://chef.local/nodes")
print(requests.get("https://chef.local"+path, headers=all_headers).text)
At each step I tried to check if the variables have the same result as their counterparts in the curl script.
The problem seems to be at signing stage - there's an obvious discrepancy between the output of python's packages and my mac's openssl tools. Due to this discrepancy, Chef returns {"error":["Invalid signature for user or client 'anton'"]}. Curl script with the same values and keys works fine.
dummy_sign = base64.b64encode(rsa.sign("hello".encode(), privkey, "SHA-1")) from Python has the value of
N7QSZRD495vV9cC35vQsDyxfOvbMN3TcnU78in911R54IwhzPUKnJTdFZ4D/KpzyTVmVBPoR4nY5um9QVcihhqTJQKy+oPF+8w61HyR7YyXZRqmx6sjiJRffC4uOGb5Wjot8csAuRSeUuHaNTl6HCcfRKnwUZnB7SctKoK6fXv0skWN2CzV9CjfHByct3oiy/xAdTz6IB+fLIwSQUf1k7lJ4/CmLJLP/Gu/qALkvWOYDAKxmavv3vYX/kNhzApKgTYPMw6l5k1aDJGRVm9Ch/BNQbg1WfZiT6LK+m4KAMFbTORfEH45KGWBCj9zsyETyMCAtUycebjqMujMqEwzv7w==
while the output of echo -n "hello" | openssl rsautl -sign -inkey ~/.chef/anton.pem | openssl enc -base64 is
WfoASF1f5DPT3CVPlWDrIiTwuEnjr5yCV+WIlbQLFmwm3nfhIqfTPLyTM56SwTSg
CKdboVU4EBFxC3RsU2aPpELqRH6+Fnl2Tl273vo6kLzvC/8+tUBTdNZdzSPhx6S8
x+6wzVFXsd3QeGAWoHkEgTKodSByFzARnZFxO2JzUe4dnygijwruHdf9S4ldrRo6
eaShwaxuNzM0cIl+Umz5iym3cCD6GFL13njmXZs3cHRLesBtLKA7pNxJ1UDf2WN2
OK09aK+bHaM4jl5HeQ2SdNzBQIKvyDcxX4Divnf2I/0tzD16J6BEMGCfTfsI2f3K
TVGulq81+sH9zo8lGnpDrw==
I couldn't find the information on default hashing algorithm in openssl for rsautl, but I guess it's SHA-1.
At this point I don't really know which way to look, hope anyone can help make it right.
How do you deal with large collections in REST api with python
I'm trying to use python to create rest API. As a framework I use flask, but I'm not using flask-restful because for one thing it doesn't follow DRY principle (IMHO).
Let's make an example.
Suppose a I have collection resource.
Products -> product -> reviews -> review
So the links would be
/products/
/products/id/
/products/id/reviews
/products/id/reviews/id
So in flask I would need to actually copy /products/ link 4 times. Flask-restful doesn't reflect that there is a hierarchy of resources. I made my own code that does that.
class ResourceInterface(object):
__metaclass__ = ABCMeta
@abstractmethod
def get_path(self):
pass
@abstractmethod
def set_path(self, path):
pass
@abstractmethod
def on_get(self, *args, **kwargs):
pass
@abstractmethod
def on_post(self, *args, **kwargs):
pass
@abstractmethod
def on_delete(self):
pass
@abstractmethod
def on_put(self):
pass
@abstractmethod
def do_put(self):
pass
@abstractmethod
def do_post(self, *args, **kwargs):
pass
@abstractmethod
def do_get(self, *args, **kwargs):
pass
@abstractmethod
def do_delete(self, *args, **kwargs):
pass
class BaseResource(ResourceInterface):
__metaclass__ = ABCMeta
def __init__(self, repository, entity_serializer, path=None):
self.entity_serializer = entity_serializer
self.repository = repository
self.path = None
self.set_path(path)
self.pickler = Pickler(unpicklable=False)
def on_put(self):
pickled_output = self.pickler.flatten(self.do_put())
return jsonpickle.json.encode(pickled_output)
def on_post(self, *args, **kwargs):
pickled_output = self.pickler.flatten(self.do_post(*args, **kwargs))
return jsonpickle.json.encode(pickled_output)
def on_get(self, *args, **kwargs):
try:
pickled_output = self.pickler.flatten(self.do_get(*args, **kwargs))
fields = request.args.get('fields')
if fields:
for field in pickled_output.keys():
if field not in fields:
del pickled_output[field]
return jsonpickle.json.encode(pickled_output)
except ResourceNotFoundError as e:
return str(e), 404
def on_delete(self, *args, **kwargs):
pickled_output = self.pickler.flatten(self.do_delete(*args, **kwargs))
return jsonpickle.json.encode(pickled_output)
def get_path(self):
return self.path
def set_path(self, path):
self.path = path
Here is a part of my code. I think it speaks for itself mostly.
The problem now is that I don't understand what is the proper way to deal with large collections. So suppose that the client doesn't want reviews data from product because it's too big.
My code is dealing with it by using 'fields' querystring, so client can say what fields exactly does he want. But that would work for only one level of hierarchy.
What to do when the client asks /products/ and doesn't want products to have reviews in the response? The concrete answer in my case is I can inspect all levels (with dfs or bfs) of response dictionary hierarchy and remove fields that wasn't in 'fields' querystring, but that's not a general answer because there could a name collisions in dictionary keys in different levels.
How does REST deals with that? How to implement this in python?
Maybe I should use cursors in all collections and always return the cursored collection and forgot about 'fields' querystring?
Or maybe some different class hierarchy needed? Like CollectionResource, ItemResource etc?
storing uncertain data in sqlalchemy
In sqlalchemy is possible to have composite columns, I do wonder if there is a ready-made mixin to support uncertainties directly.
Alternatively using pandas as storage would be a suggested alternative.
Regex Greediness
it might be hard question related regular expression but I couldn't solve it. Here is my regular expression:
regex = (^|(?<= ))Football( ((\S+ )+?(?=Football)|(\S+ )+)| )fun( ((\S+ )+?(?=Football)|(\S+ )+)| )Football\ is\ important((?= )|$)
With that I'd like to catch this:
text1 = "Football is fun I like Football is important"
but not this:
text2 = "Football is fun I like Football Football is important"
As far as I understand, expression shouldn't have matched because there is one more Football in there. Second ( ((\S+ )+?(?=Football)|(\S+ )+)| ) part should have matched I like because after this Football in there and it's not greedy because I added ? after second +. The last part should have matched Football is important so there is one Football (in the middle) hanging around. How can I modify it so that it makes what I need?
Sorry for the silly example; I changed my real text.
Mix Python virtualenv packages with distro packages?
What is a good way to handle the case when you use Python virtualenv but you want some of the packages installed via your distro's package manager?
Let's say you need lxml but because you can't get pip install lxml to work on Ubuntu. and you really don't want to waste time on this, so you just do a apt-get install python-lxml.
Now, you can create a virtaulenv with --system-site-packages and have access to the system-wide installed pre-compiled lxml now. But you'll also drag in all the other system wide packages that you don't need! And yes, there will be quite a bunch of packages that will be installed outside virtualenv, either via sudo pip ... or sudo apt-get python-..., so no "just keep the system clean and install everything you can in virtualenvs, so that --system-site-packages won't drag too many packages with it" is not a solution form me here.
So, is there a way to install just some particular system-site-packages?
Match two SQL database records using Python by a database column field
I got a problem when i tried to compare two databases that i read using python. The problem is to write proper for cycle. Databases are read using:
output_po = psql.read_frame(sql_po, conn) output_stock = psql.read_frame(sql_stock, conn)
SQL_PO records: PO PN
FTP1 1111
FTP2 2222
SQL_STOCK records PN WHS NAME 1111 VLN A 1111 VLN B 1111 ZRS A 2222 DLN Q
So the result i want to get is to get all SQL_STOCK info based on SQL_PO info where Whs == 'VLN'. Result: PO PN WHS NAME FTP1 1111 VLN A FTP1 1111 VLN B
I have tried using:
for index, row in sql_po.iterrows():
if sql_stock[sql_stock['PN']==row['PN']].empty:
send_email = 'F'
else:
if (sql_stock[sql_stock['Whs'] =='VLN'].any ):
send_email = 'T'
But it gives all 3 records for PN 1111 from sql_stock as for 1111 there are 2 with vln and 1 with zrs whs. COuld you guys help me solve that problem to write cycle for in another cycle for?
PyMongo, handling fields with time relative to the database time
Say I have a document field name "creation_time". I would like to have its value initialized with the database current time, instead using the client machine time with:
datetime.datetime.utcnow()
How can I achieve this with PyMongo?
Moreover is there any way to do operations during the field initialization, for example:
"creation_time": magic_get_mongodb_time() + 10 hours
As a side note I'm OK to handle unix timestamps, not necessary datetime format.
EDIT: operations on dates are not possible yet (May 2015): http://ift.tt/1FTC07K
Facing error while starting map job.
While starting a map job i am getting this error.
TypeError: __init__() got an unexpected keyword argument _user_agent
Here is how i am starting the map function.
control.start_map(name='Export Device Health Logs',
handler_spec='data_process.process_health_logs',
reader_spec='mapreduce.input_readers.DatastoreInputReader',
mapper_parameters={"input_reader":{'entity_kind': 'models.DeviceHealth','email':user.email,'to_date':to_date,'from_date':from_date},'output_writer': {'bucket_name': bucket_name,
'content_type': 'text/plain'}},
shard_count=1,
output_writer_spec='mapreduce.output_writers.GoogleCloudStorageConsistentOutputWriter',
mapreduce_parameters={'done_callback':'/tasks/mapreduce/done/export_health_data',
'done_callback_queue':'mapreduce-done'})
Regex to segregate list of strings and regexs
I have python input list containing both strings and regexs.
str_regex = ['normal_string_1', ''^(?![_\-])[A-Za-z0-9\-_]+$', 'normal_string_2']
I need to segregate this list into list of strings and list of regexs. In summary below are input and required output.
Input :
['normal_string_1', ''^(?![_\-])[A-Za-z0-9\-_]+$', 'normal_string_2']
Output :
['normal_string_1', 'normal_string_2'], ['^(?![_\-])[A-Za-z0-9\-_]+$']
I am planning to form a regex to do this. Something like "Should contains only [A-Z] and [a-z]". Is this the best way to do it?
How to emit websocket message from outside a websocket endpoint?
I'm building a website using Flask in which I also use Websockets using Flask-socketIO, but there's one thing I don't understand.
I built a chat-functionality. When one user sends a message I use websockets to send that message to the server, after which I emit the message to the other user from within that same call:
@socketio.on('newPM', namespace='/test')
@login_required_with_global
def io_newMessage(theJson):
emit('message', {'message': theJson['message']}, room=str(theJson['toUserId']))
But let's say that I want to emit a message to a user when a file was saved. This means that I need to emit a message from within the view in which the file is POSTed. So according to the flask_socketio docs I can add a namespace in the emit. So I wrote this:
@app.route('/doc', methods=['POST'])
@login_required
def postDoc():
saveDocument(request.files['file'], g.user.id)
emit('my response', {'data': 'A NEW FILE WAS POSTED'}, room=current_user.id, namespace='/test')
return jsonify({'id': str(doc.id)})
But seeing the stacktrace below there still is a problem with the namespace; werkzeug has an AttributeError: 'Request' object has no attribute 'namespace'.
Does anybody know what I'm doing wrong here? Or is this a bug in flask_socketio? All tips are welcome!
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1820, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1403, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1817, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1477, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1381, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1475, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1461, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/local/lib/python2.7/dist-packages/flask_login.py", line 758, in decorated_view
return func(*args, **kwargs)
File "/home/vg/app/views.py", line 768, in emitNotificationCount
emit('menuInit', emitJson, room=current_user.id, namespace='/test')
File "/usr/local/lib/python2.7/dist-packages/flask_socketio/__init__.py", line 444, in emit
return request.namespace.emit(event, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/werkzeug/local.py", line 338, in __getattr__
return getattr(self._get_current_object(), name)
AttributeError: 'Request' object has no attribute 'namespace'
using cython to generate a unix timestamp
If you have every integer in memory to construct a datetime object, is there a better way than the following.
atoi(datetime(year,month,day,hour,minute,second).stftime("%s"))
searching for gedit python console commands
I am searching for a command list for the python console in gedit. This would be for example like a print statement, manipulating and processing text or the like.
Thanks.
Python convert binary into string while ignoring non-ascii characters
I have a binary file and I want to extract all ascii characters while ignoring non-ascii ones. Currently I have:
with open(filename, 'rb') as fobj:
text = fobj.read().decode('utf-16-le')
file = open("text.txt", "w")
file.write("{}".format(text))
file.close
However I'm encountering an error when writing to file UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0' in position 0: ordinal not in range(128). How would I get Python to ignore non-ascii?
convert unsigned integer to float in python
I wrote a socket server that reads data from some devices. After reading data binary shift is applied on bytes. After that i get an integer value for instance 1108304047 and i want to convert this number to IEEE 754 float 35.844417572021484. I found some solutions with struct.unpack but it doesn't seem to me rational. First we convert number to string then convert to float.
Is there any short way like Float.intBitsToFloat(1108304047) in Java.
pdb.set_trace(): script stops at the breakpoint, but doesn't show me a prompt
I have a script that runs a brian2 neural-network simulation. Recently, the script has been producing warnings that originate deep in scipy, in a module called _dumbdbm_patched. In order to debug this issue, I've inserted
import pdb
pdb.set_trace()
into the scipy source code installed on my computer. When I run my script in IPython, it stops at the breakpoint I've set, but the (Pdb) prompt never displays. Only after I've executed a keyboard interrupt does the prompt reveal itself -- just before the KeyboardInterrupt exits the program:
In [2]: run ardid.py --k 1 --g fig_4
^C> /home/despo/dbliss/lib/python2.7/site-packages/scipy/weave/_dumbdbm_patched.py(163)open()
-> return _Database(file)
(Pdb) ---------------------------------------------------------------------------
KeyboardInterrupt Traceback (most recent call last)
/home/despo/dbliss/dopa_net/brian/ardid/ardid.py in <module>()
173 # -------------------------------------------------------------------------
174
--> 175 cii_pfc = brian2.Synapses(pi_pfc, pi_pfc, pre='s_gaba += 1', connect=True)
176 cie_pfc = brian2.Synapses(pi_pfc, pe_pfc, pre='s_gaba += 1', connect=True)
177
/home/despo/dbliss/lib/python2.7/site-packages/Brian2-2.0b3+git-py2.7-linux-x86_64.egg/brian2/synapses/synapses.pyc in __init__(self, source, target, model, pre, post, connect, delay, namespace, dtype, codeobj_class, dt, clock, order, method, name)
756
757 if not connect is False:
--> 758 self.connect(connect, level=1)
759
760 def __len__(self):
/home/despo/dbliss/lib/python2.7/site-packages/Brian2-2.0b3+git-py2.7-linux-x86_64.egg/brian2/synapses/synapses.pyc in connect(self, pre_or_cond, post, p, n, namespace, level)
1064 'to an float, is type %s instead.' % type(n))
1065 self._add_synapses(None, None, n, p, condition=pre_or_cond,
-> 1066 namespace=namespace, level=level+1)
1067 else:
1068 raise TypeError(('First argument has to be an index or a '
/home/despo/dbliss/lib/python2.7/site-packages/Brian2-2.0b3+git-py2.7-linux-x86_64.egg/brian2/synapses/synapses.pyc in _add_synapses(self, sources, targets, n, p, condition, namespace, level)
1192 check_units=False,
1193 run_namespace=namespace,
-> 1194 level=level+1)
1195 codeobj()
1196
/home/despo/dbliss/lib/python2.7/site-packages/Brian2-2.0b3+git-py2.7-linux-x86_64.egg/brian2/codegen/codeobject.pyc in create_runner_codeobj(group, code, template_name, user_code, variable_indices, name, check_units, needed_variables, additional_variables, level, run_namespace, template_kwds, override_conditional_write, codeobj_class)
256
257 if codeobj_class is None:
--> 258 codeobj_class = device.code_object_class(group.codeobj_class)
259 else:
260 codeobj_class = device.code_object_class(codeobj_class)
/home/despo/dbliss/lib/python2.7/site-packages/Brian2-2.0b3+git-py2.7-linux-x86_64.egg/brian2/devices/device.pyc in code_object_class(self, codeobj_class)
199 def code_object_class(self, codeobj_class=None):
200 if codeobj_class is None:
--> 201 codeobj_class = get_default_codeobject_class()
202 return codeobj_class
203
/home/despo/dbliss/lib/python2.7/site-packages/Brian2-2.0b3+git-py2.7-linux-x86_64.egg/brian2/devices/device.pyc in get_default_codeobject_class(pref)
77 if isinstance(codeobj_class, str):
78 if codeobj_class == 'auto':
---> 79 return auto_target()
80 for target in codegen_targets:
81 if target.class_name == codeobj_class:
/home/despo/dbliss/lib/python2.7/site-packages/Brian2-2.0b3+git-py2.7-linux-x86_64.egg/brian2/devices/device.pyc in auto_target()
51 if target.class_name)
52 using_fallback = False
---> 53 if 'weave' in target_dict and target_dict['weave'].is_available():
54 _auto_target = target_dict['weave']
55 elif 'cython' in target_dict and target_dict['cython'].is_available():
/home/despo/dbliss/lib/python2.7/site-packages/Brian2-2.0b3+git-py2.7-linux-x86_64.egg/brian2/codegen/runtime/weave_rt/weave_rt.pyc in is_available()
135 headers=['<algorithm>', '<limits>'],
136 extra_compile_args=extra_compile_args,
--> 137 verbose=0)
138 return True
139 except Exception as ex:
/home/despo/dbliss/lib/python2.7/site-packages/scipy/weave/inline_tools.py in inline(code, arg_names, local_dict, global_dict, force, compiler, verbose, support_code, headers, customize, type_converters, auto_downcast, newarr_converter, **kw)
350 # 2. try function catalog
351 try:
--> 352 results = attempt_function_call(code,local_dict,global_dict)
353 # 3. build the function
354 except ValueError:
/home/despo/dbliss/lib/python2.7/site-packages/scipy/weave/inline_tools.py in attempt_function_call(code, local_dict, global_dict)
419 # 3. try persistent catalog
420 module_dir = global_dict.get('__file__',None)
--> 421 function_list = function_catalog.get_functions(code,module_dir)
422 for func in function_list:
423 try:
/home/despo/dbliss/lib/python2.7/site-packages/scipy/weave/catalog.py in get_functions(self, code, module_dir)
793 try:
794 self.set_module_directory(module_dir)
--> 795 function_list = self.get_cataloged_functions(code)
796 # put function_list in cache to save future lookups.
797 if function_list:
/home/despo/dbliss/lib/python2.7/site-packages/scipy/weave/catalog.py in get_cataloged_functions(self, code)
708 function_list = []
709 for path in self.build_search_order():
--> 710 cat = get_catalog(path,mode)
711 if cat is not None and code in cat:
712 # set up the python path so that modules for this
/home/despo/dbliss/lib/python2.7/site-packages/scipy/weave/catalog.py in get_catalog(module_path, mode)
479 and ((dumb and os.path.exists(catalog_file+'.dat'))
480 or os.path.exists(catalog_file)):
--> 481 sh = shelve.open(catalog_file,mode)
482 else:
483 if mode == 'r':
/home/despo/dbliss/lib/python2.7/site-packages/scipy/weave/_dumb_shelve.py in open(filename, flag)
49 """
50
---> 51 return DbfilenameShelf(filename, flag)
/home/despo/dbliss/lib/python2.7/site-packages/scipy/weave/_dumb_shelve.py in __init__(self, filename, flag)
20 def __init__(self, filename, flag='c'):
21 from . import _dumbdbm_patched
---> 22 Shelf.__init__(self, _dumbdbm_patched.open(filename, flag))
23
24 def __getitem__(self, key):
/home/despo/dbliss/lib/python2.7/site-packages/scipy/weave/_dumbdbm_patched.py in open(file, flag, mode)
161 pdb.set_trace()
162
--> 163 return _Database(file)
/home/despo/dbliss/lib/python2.7/site-packages/scipy/weave/_dumbdbm_patched.py in open(file, flag, mode)
161 pdb.set_trace()
162
--> 163 return _Database(file)
/usr/local/anaconda-1.9.2/lib/python2.7/bdb.pyc in trace_dispatch(self, frame, event, arg)
47 return # None
48 if event == 'line':
---> 49 return self.dispatch_line(frame)
50 if event == 'call':
51 return self.dispatch_call(frame, arg)
/usr/local/anaconda-1.9.2/lib/python2.7/bdb.pyc in dispatch_line(self, frame)
65 def dispatch_line(self, frame):
66 if self.stop_here(frame) or self.break_here(frame):
---> 67 self.user_line(frame)
68 if self.quitting: raise BdbQuit
69 return self.trace_dispatch
/usr/local/anaconda-1.9.2/lib/python2.7/pdb.pyc in user_line(self, frame)
156 self._wait_for_mainpyfile = 0
157 if self.bp_commands(frame):
--> 158 self.interaction(frame, None)
159
160 def bp_commands(self,frame):
/usr/local/anaconda-1.9.2/lib/python2.7/pdb.pyc in interaction(self, frame, traceback)
208 self.setup(frame, traceback)
209 self.print_stack_entry(self.stack[self.curindex])
--> 210 self.cmdloop()
211 self.forget()
212
/usr/local/anaconda-1.9.2/lib/python2.7/cmd.pyc in cmdloop(self, intro)
128 if self.use_rawinput:
129 try:
--> 130 line = raw_input(self.prompt)
131 except EOFError:
132 line = 'EOF'
KeyboardInterrupt:
The same thing happens when I make the call directly from the shell, as in
$ python ardid.py --k 1 --g fig_4
Why is this happening? How can I get pdb to work inside scipy this way?
Note: Nothing in my script is threaded.
Python subprocess: pipe an image blob to imagemagick shell command
I have an image in-memory and I wish to execute the convert method of imagemagick using Python's subprocess. While this line works well using Ubuntu's terminal:
cat image.png | convert - new_image.jpg
This piece of code doesn't work using Python:
jpgfile = Image.open('image.png');
proc = Popen(['convert', '-', 'new_image.jpg'], stdin=PIPE, shell=True)
print proc.communicate(jpgfile.tostring())
I've also tried reading the image as a regular file without using PIL, I've tried switching between subprocess methods and different ways to write to stdin.
The best part is, nothing is happening but I'm not getting a real error. When printing stdout I can see imagemagick help on terminal, followed by the following:
By default, the image format of `file' is determined by its magic number. To specify a particular image format, precede the filename with an image format name and a colon (i.e. ps:image) or specify the image type as the filename suffix (i.e. image.ps). Specify 'file' as '-' for standard input or output. (None, None)
Maybe there's a hint in here I'm not getting. Please point me in the right direction, I'm new to Python but from my experience with PHP this should be an extremely easy task, or so I hope.
Python: How to read a directory of texts into a list
I am trying to work with gensim for topic modeling. From what I can tell looking at the module's documentation, gensim expects to receive its input as a list, with each item in a list being a text:
documents = ["Human machine interface for lab abc computer applications",
"A survey of user opinion of computer system response time",
"The EPS user interface management system"]
I have a collection of texts in a directory that I would like to use with gensim, and so I need to read those files into a list. Each of those texts, some of which consist of multiple lines -- the texts range in size from a little under 100 words to a little over 1000 words -- needs to be one item in the list. If stripping newlines out is required, I think I can figure out how to do that, but embedding it into a loop is where I fail ... completely. (In fact, I am taking myself to loop school over the weekend, but I regularly mess that part up.)
I have found all kinds of useful information on how to read a single file into a list -- by line or by word or by whatever -- but I can't figure out how to read a series of text files into a series of strings all contained within a single list -- this is the important bit:
textfile1.txt
textfile2.txt
need to become
list = ['contents of textfile1', 'contents of textfile2']
Here's what I have so far:
# get to the files, open an empty list
import glob
file_list = glob.glob('./texts' + '/*.txt')
documents = []
# Now to read the files into a list:
for file in file_list:
documents.append()
print documents
The print documents is obviously a throwaway line so I could check my work, and you can see that I didn't get very far with the loop.
python just import fails but works with from
Why does this not work:
import matplotlib.pyplot as plt
import os
import skimage
camera = skimage.io.imread(os.path.join(skimage.data_dir, 'camera.png'))
#plt.show(io.imshow(camera))
But using from skimage import io does. So this works:
import matplotlib.pyplot as plt
import os
import skimage # I still need to import skimage to get the data_dir
from skimage import io
camera = io.imread(os.path.join(skimage.data_dir, 'camera.png'))
#plt.show(io.imshow(camera))
I thought
import skimage
skimage.io."something"
Was equivalent to
from skimage import io
io."something"
Using while or if as a condition for recursion leads to different results
This is a simple recursive function that should run 10 times, the condition is if.
count = 0
def recurse(count):
*if* count < 10:
print count
count += 1
recurse(count)
recurse(count)
Output
0 1 2 3 4 5 6 7 8 9 OK
When I use a while loop, the results are wildly different and I don't understand why it does not output 0 to 9.
Code
count = 0
def recurse(count):
*while* count < 10:
print count
count += 1
recurse(count)
recurse(count)
Output
0 1 2 3 4 5 6 7 8 9 9 8 9 9 7 8 9 9 8 9 9 6 7 8 9 9 8 9 9 7..... 8 9 9
You can try it here http://ift.tt/XiergB, I don't know how to create a link with the code though.
anyone see what I am doing wrong.
Edit.
Output of code 2 is finite.
Example using 3 as a limit.
- using if
0 1 2 - using while
0 1 2 2 1 2 2
Why isn't the square moving when a key is pressed?
Im trying to code a tron bike game and right now I am coding the part to get the square to move. I'm still a little bit of a newbie at coding, so my code might have some problems that I dont know yet. Here is my code:
import sys, pygame, math, random, itertools
from pygame.locals import *
from MyLibrary import *
def game_init():
global screen,backbuffer,font,timer,player_group,player, \
enemy_tank,bullets,crosshair,crosshair_group
pygame.init()
screen = pygame.display.set_mode((800,600))
backbuffer = pygame.Surface((800,600))
pygame.display.set_caption("Tron Bikes")
font = pygame.font.Font(None,30)
timer = pygame.time.Clock()
def audio_init():
global shoot_sound, boom_sound
pygame.mixer.init()
boom_sound = pygame.mixer.Sound("bomb.wav")
def play_sound(sound):
channel = pygame.mixer.find_channel(True)
channel.set_volume(50)
channel.play(sound)
game_init()
audio_init()
game_over = False
round = 0
player_score = 0
enemy_score = 0
x = 600
y = 450
x1 = 200
y1 = 150
pygame.draw.rect(backbuffer, (0,0,255),(x,y,20,20),0)
screen.blit(backbuffer,(0,0))
backbuffer.fill((100,100,100))
while True:
timer.tick(30)
ticks = pygame.time.get_ticks()
for event in pygame.event.get():
if event.type == QUIT:
sys.exit()
keys = pygame.key.get_pressed()
if keys[K_w]:
y -= 1
if keys[K_d]:
x += 1
if keys[K_s]:
y += 1
if keys[K_a]:
x -= 1
if keys[K_ESCAPE]:
sys.exit()
if x > 800:
x = 800
if x < 0:
x = 0
if y > 600:
y = 600
if y < 0:
y = 0
pygame.display.update()
EDIT: Forgot to say that I am making a tron bike game, so when w,a,s,d is pressed, im trying to make it go that way until another key is pressed, which is where I can't figure it out.
AttributeError: 'tuple' object has no attribute 'strip'
I use python and this is my code
url = 'https://api.url.net', "{\"orga\":\"monorga\",\"coupon\":\"moncoupon\"}"
url = url.strip()
AttributeError: 'tuple' object has no attribute 'strip'
Please help me
How to correctly break a long line in Python?
How would I go about breaking the following line? The PEP8 guideline doesn't make it very clear to me.
confirmation_message = _('ORDER_CREATED: %(PROPERTY_1)s - %(PROPERTY_2)s - %(PROPERTY_3)s - %(PROPERTY_4)s') % {'PROPERTY_1': order.lorem, 'PROPERTY_2': order.ipsum, 'PROPERTY_4': order.dolor, 'PROPERTY_5': order.sit}
numpy arange values are changing signs unexpectedly
I am teaching mysef a little about numpy and I have dusted off some of my old undergraduate texts to use for examples. So, I wrote a function without numpy to calculate the deflection in a cantilever beam due to a single point load at any point. Pretty straight forward, except that the deflection equation changes depending on what side of the point force you are on, so I will split the beam into two ranges, calculate the deflection values at each interval in the ranges and append the result to a list. Here's the code.
def deflection(l, P, a, E, I):
"""
Calculate the deflection of a cantilever beam due to a simple point load.
Calculates the deflection at equal one inch intervals along the beam and
returns the deflection as a list along with the the length range.
Parameters
----------
l : float
beam length (in)
P : float
Point Force (lbs)
a : float
distance from fixed end to force (in)
E : float
modulus of elasticity (psi)
I : float
moment of inertia (in**4)
Returns
-------
list
x : distance along beam (in)
list of floats
y : deflection of beam (in)
Raises
------
ValueError
If a < 0 or a > l (denoting force location is off the beam)
"""
if (a < 0) or (a > l):
raise ValueError('force location is off beam')
x1 = range(0, a)
x2 = range(a, l + 1)
deflects = []
for x in x1:
y = (3 * a - x) * (P * x**2) / (6 * E * I)
deflects.append(y)
for x in x2:
y = (3 * x - a) * (P * a**2) / (6 * E * I)
deflects.append(y)
return list(x1) + list(x2), deflects
Now I want to do the same thing using numpy, so I wrote the following function:
def np_deflection(l, P, a, E, I):
"""To Do. Write me."""
if (a < 0) or (a > l):
raise ValueError('force location is off beam')
x1 = np.arange(0, a)
x2 = np.arange(a, l + 1)
y1 = (3 * a - x1) * (P * x1**2) / (6 * E * I)
y2 = (3 * x2 - a) * (P * a**2) / (6 * E * I)
return np.hstack([x1, x2]), np.hstack([y1, y2])
Here's the issue, at some point in the calculations, the value of y1 changes sign. Here's an example.
if __name__ == '__main__':
import matplotlib.pyplot as plt
l, P, a, E, I = 120, 1200, 100, 30000000, 926
x, y = deflection(l, P, a, E, I)
print(max(y))
np_x, np_y = np_deflection(l, P, a, E, I)
print(max(np_y))
plt.subplot(2, 1, 1)
plt.plot(x, y, 'b.-')
plt.xlabel('dist from fixed end (in)')
plt.ylabel('using a range/list')
plt.subplot(2, 1, 2)
plt.plot(np_x, np_y, 'r.-')
plt.xlabel('dist from fixed end (in)')
plt.ylabel('using numpy range')
plt.show()
If you run the plot, you'll see that at point x = 93 which is in x1, there is a dislocation in the curve where the value seems to change sign.
Inscription à :
Articles (Atom)