| Home | Forums | Reviews | Guides | Newsgroups | Register | Search |
![]() |
| Thread Tools |
| Jack Carter |
|
|
|
| |
|
Peter Otten
Guest
Posts: n/a
|
Jack Carter wrote:
> This will result in: > > >>> bosco=5 > >>> if 1: > ... attach bosco > ... bosco=7 > ... attach bosco > ... > DoAttach: ['5'] > DoAttach: ['5'] <--- WRONG (at least, not what I want) > >>> attach bosco > DoAttach: ['7'] The suite attach bosco bosco = 7 attach bosco is only executed after the final empty line. Therefore any access to the right dictionary in the process() method will give you the current value bosco==5 which is not what you want. You are in effect translating the suite into (simplified) DoAttach(5) bosco = 7 DoAttach(5) # the previous line has not yet been executed I think the easiest way to get the desired effect ("late binding") is like so: def process(self, line): temp_line = line.lstrip() front_padding = line[:len(line)-len(temp_line)] match = self.regexp.match(temp_line) if match: line = front_padding + "myparse.DoAttach(bosco)" return line I. e. change the generated line to contain variable names instead of values and leave the resolution to python. (You hint you don't wont that either. Why?) By the way, I wasn't able to run your code - the indentation is seriously messed up. I recommend you ensure a 4-space indentation in all code you currently have before you move on. That will spare you a lot of trouble later on. Another minor issue: 'value is 0' may or may not work depending on the python implementation ('value is 1000' won't work even in current CPython). With 'value == 0' you are on the safe side. Peter |
|
|
|
|
|||
|
|||
| Peter Otten |
|
|
|
| |
|
Jack Carter
Guest
Posts: n/a
|
Peter,
I guess I just don't understand. Basically this is to go into a tool that accepts commandline arguments, one at a time, but will also allow scripting for testing purposes. Thus the desire to leverage off python for the commandline interface, but not for the whole program. The input from the user will be either from the console or read from a command file. A simple example of what could be entered is: bosco=5 if 1: print bosco attach bosco 9 bosco=7 print bosco attach bosco 9 The result I would expect would be for my DoAttach() routine to receive the python evaluated value of each of the arguments leaving the ones that it doesn't understand alone. Whether this happens automagically or by hand I don't care as long as I get what right value. In the above trying to follow your advice, remembering that at this stage of the game I am probably missing the point, this is the result I expect: johmar % demo.py >>> bosco=5 >>> if 1: .... print bosco .... attach bosco 9 .... bosco=7 .... print bosco .... attach bosco 9 .... 5 DoAttach: [5, 9] 7 DoAttach: [7, 9] This is what I get: johmar % demo.py >>> bosco=5 >>> if 1: .... print bosco .... attach bosco 9 .... bosco=7 .... print bosco .... attach bosco 9 .... 5 DoAttach: ['bosco', '9'] 7 DoAttach: ['bosco', '9'] Now I realize that this is probably due to the fact that I have the lines: if white_spaces: line = front_padding + "myparse." + function + "(" + str(args) + ")" else : line = "myparse." + function + "(" + str(args) + ")" which put make the arguments strings, but that is because I don't know how to appropriately pack the "line" for later parsing. Maybe that is the crux of my problem. Remember, there will be many commands for my tool and the arguments will be variable length and this code will not know what variables the gentle use would use. Here is the simplified code with hopefully the tabs expanded base on your earlier input. Hopefully you'll see the obvious error of my way and point it out. Thanks ever so much, Jack ********************************** demo.py ********************************** #!/usr/bin/env python import myparse cli = myparse.CLI(globals()) cli.interact() ********************************** myparse.py ********************************** import code import re import string import sys ################################################## ############################## # # DoAttach # # Dummy function that I will eventually use to do # real stuff. # ################################################## ############################## def DoAttach(args): print "DoAttach:", args pass class CLI(code.InteractiveConsole): """Simple test of a Python interpreter augmented with custom commands.""" commands = { \ "attach" : "DoAttach" } def __init__(self, locals = None): # Call super-class initializer code.InteractiveConsole.__init__(self, locals, "<console>") # Compile regular expression for finding commmands self.regexp = re.compile('[a-z]*') ################################################## ################ # # interact # # This will read and process input lines from within # my main application as though on a python commandline. # ################################################## ################ def interact(self): # Set the primary and secondary prompts sys.ps1 = ">>> " sys.ps2 = "... " # Input Loop is_more = 0 bosco = 0 while 1: try : # Display the appropriate prompt if not sys.stdin.isatty(): prompt = "" elif is_more: prompt = sys.ps2 else: prompt = sys.ps1 # Read the next line of input #self.write("interact 1\n") line = self.raw_input(prompt) # TODO: add logging of input line here... # Process complete lines if 1: line = self.process(line) # Push incomplete lines onto input stack if line or is_more: is_more = self.push(line) # Handle CTRL-C except KeyboardInterrupt: self.write("\nKeyboardInterrupt\n") is_more = 0 self.resetbuffer() # Handle CTRL-D except EOFError: self.write("\n") is_more = 0 self.resetbuffer() raise SystemExit ################################################## ################ # # process # # This will determine if the input command is either # from my application's command language or a python # construct. # ################################################## ################ def process(parent, line): # Attempt to match line against our command regular expression temp_line = string.lstrip(line) len_1 = len(line) len_2 = len(temp_line) white_spaces = len_1-len_2 if white_spaces: front_padding = line[0:white_spaces] match = parent.regexp.match(temp_line) if match is not None: #parent.write("process 1\n") # Extract the command and argument strings cmd_string = match.group() arg_string = string.lstrip(temp_line[match.end():]) # Find the function for this command in the command dictionary function = parent.commands.get(cmd_string) if function is not None: # Split argument string into individual arguments args = string.split(arg_string) # Convert to Python function-call syntax for this command if white_spaces: line = front_padding + "myparse." + function + "(" + str(args) + ")" else : line = "myparse." + function + "(" + str(args) + ")" # Return the line to be processed by Python return line |
|
|
|
|
|||
|
|||
| Jack Carter |
|
Peter Otten
Guest
Posts: n/a
|
Jack Carter wrote:
> Basically this is to go into a tool that accepts > commandline arguments, one at a time, but will also > allow scripting for testing purposes. Thus the desire > to leverage off python for the commandline interface, > but not for the whole program. > > The input from the user will be either from the console or > read from a command file. A simple example of what could > be entered is: > > bosco=5 > if 1: > print bosco > attach bosco 9 > bosco=7 > print bosco > attach bosco 9 > > The result I would expect would be for my DoAttach() > routine to receive the python evaluated value of each > of the arguments leaving the ones that it doesn't understand > alone. Whether this happens automagically or by hand I don't > care as long as I get what right value. > > In the above trying to follow your advice, remembering that > at this stage of the game I am probably missing the point, this > is the result I expect: > > johmar % demo.py >>>> bosco=5 >>>> if 1: > ... print bosco > ... attach bosco 9 > ... bosco=7 > ... print bosco > ... attach bosco 9 > ... > 5 > DoAttach: [5, 9] > 7 > DoAttach: [7, 9] > > This is what I get: > > johmar % demo.py >>>> bosco=5 >>>> if 1: > ... print bosco > ... attach bosco 9 > ... bosco=7 > ... print bosco > ... attach bosco 9 > ... > 5 > DoAttach: ['bosco', '9'] > 7 > DoAttach: ['bosco', '9'] > > Now I realize that this is probably due to the fact that > I have the lines: > > if white_spaces: > line = front_padding + "myparse." + function + "(" + > str(args) + ")" > else : > line = "myparse." + function + "(" + str(args) + ")" > > which put make the arguments strings, but that is because I don't > know how to appropriately pack the "line" for later parsing. Maybe > that is the crux of my problem. Remember, there will be many commands > for my tool and the arguments will be variable length and this code > will not know what variables the gentle use would use. > > Here is the simplified code with hopefully the tabs expanded > base on your earlier input. Hopefully you'll see the obvious > error of my way and point it out. > > Thanks ever so much, > > Jack > > ********************************** > demo.py > ********************************** > #!/usr/bin/env python > > import myparse > > cli = myparse.CLI(globals()) > cli.interact() > > ********************************** > myparse.py > ********************************** > import code > import re > import string > import sys > > ################################################## ############################## > # > # DoAttach > # > # Dummy function that I will eventually use to do > # real stuff. > # > ################################################## ############################## > def DoAttach(args): > > print "DoAttach:", args > pass > > > class CLI(code.InteractiveConsole): > """Simple test of a Python interpreter augmented with custom > commands.""" > > commands = { \ > "attach" : "DoAttach" > } > > def __init__(self, locals = None): > > # Call super-class initializer > code.InteractiveConsole.__init__(self, locals, "<console>") > > # Compile regular expression for finding commmands > self.regexp = re.compile('[a-z]*') > > > ################################################## ################ > # > # interact > # > # This will read and process input lines from within > # my main application as though on a python commandline. > # > ################################################## ################ > def interact(self): > > # Set the primary and secondary prompts > sys.ps1 = ">>> " > sys.ps2 = "... " > > # Input Loop > is_more = 0 > bosco = 0 > while 1: > try : > # Display the appropriate prompt > if not sys.stdin.isatty(): > prompt = "" > elif is_more: > prompt = sys.ps2 > else: > prompt = sys.ps1 > > # Read the next line of input > #self.write("interact 1\n") > line = self.raw_input(prompt) > > # TODO: add logging of input line here... > > # Process complete lines > if 1: > line = self.process(line) > > # Push incomplete lines onto input stack > if line or is_more: > is_more = self.push(line) > > # Handle CTRL-C > except KeyboardInterrupt: > self.write("\nKeyboardInterrupt\n") > is_more = 0 > self.resetbuffer() > > # Handle CTRL-D > except EOFError: > self.write("\n") > is_more = 0 > self.resetbuffer() > raise SystemExit > > ################################################## ################ > # > # process > # > # This will determine if the input command is either > # from my application's command language or a python > # construct. > # > ################################################## ################ > def process(parent, line): > > # Attempt to match line against our command regular expression > > temp_line = string.lstrip(line) > len_1 = len(line) > len_2 = len(temp_line) > > white_spaces = len_1-len_2 > if white_spaces: > front_padding = line[0:white_spaces] > > match = parent.regexp.match(temp_line) > if match is not None: > > #parent.write("process 1\n") > # Extract the command and argument strings > cmd_string = match.group() > arg_string = string.lstrip(temp_line[match.end():]) > > # Find the function for this command in the command dictionary > function = parent.commands.get(cmd_string) > > if function is not None: > > # Split argument string into individual arguments > args = string.split(arg_string) > > # Convert to Python function-call syntax for this command > if white_spaces: > line = front_padding + "myparse." + function + "(" + > str(args) + ")" > else : > line = "myparse." + function + "(" + str(args) + ")" > # let's add some feedback print "fed to the snake:", line > # Return the line to be processed by Python > return line Now >>> bosco = 1 fed to the snake: bosco = 1 >>> attach bosco 2 fed to the snake: myparse.DoAttach(['bosco', '2']) DoAttach: ['bosco', '2'] >>> You build a function call myparse.DoAttach(['bosco', '2']) But what you need would rather be myparse.DoAttach([bosco, 2]) To achieve that you have to somehow extract (Python-compatible) expressions for the arguments given in the line attach bosco 2 which can be arbitrarily complex depending on how you defined your custom language. Assumming that you use the simplest possible spec, a space-separated list of already Python-compatible expressions that gives you def parseArgs(args): return args.split() def makePythonCall(func, args): return "%s([%s])" % (func, ", ".join(args)) and the process() method will become: def process(parent, line): # Attempt to match line against our command regular expression temp_line = string.lstrip(line) len_1 = len(line) len_2 = len(temp_line) white_spaces = len_1-len_2 if white_spaces: front_padding = line[0:white_spaces] match = parent.regexp.match(temp_line) if match is not None: #parent.write("process 1\n") # Extract the command and argument strings cmd_string = match.group() arg_string = string.lstrip(temp_line[match.end():]) # Find the function for this command in the command dictionary function = parent.commands.get(cmd_string) if function is not None: args = parseArgs(arg_string) line = makePythonCall("myparse." + function, args) if white_spaces: line = front_padding + line print "fed to the snake:", line # Return the line to be processed by Python return line Testing it: >>> bosco = 1 fed to the snake: bosco = 1 >>> attach bosco 2 fed to the snake: myparse.DoAttach([bosco, 2]) DoAttach: [1, 2] >>> if 1: fed to the snake: if 1: .... attach bosco fed to the snake: myparse.DoAttach([bosco]) .... bosco = 3 fed to the snake: bosco = 3 .... attach bosco fed to the snake: myparse.DoAttach([bosco]) .... fed to the snake: DoAttach: [1] DoAttach: [3] Works here, but is not very robust: >>> attach = 99 fed to the snake: myparse.DoAttach([=, 99]) File "<console>", line 1 myparse.DoAttach([=, 99]) ^ SyntaxError: invalid syntax >>> attach(bosco) fed to the snake: myparse.DoAttach([(bosco)]) DoAttach: [3] >>> attach bosco "so what" fed to the snake: myparse.DoAttach([bosco, "so, what"]) DoAttach: [3, 'so, what'] # note the comma Peter |
|
|
|
|
|||
|
|||
| Peter Otten |
|
Jack Carter
Guest
Posts: n/a
|
Peter,
Actually although your solution is a good one, it won't really help me because of the nature of our command language. We have commands in the form of: result = <command_name> <filename> -e <experiment_name> I currently scan ahead for <command_name> and could do so for the rest to prevent them from being evaluated by python by packing them with quotes and then unpacking them later. The problem lies in the case where the name being used in the commandline may or may not be a formal name such as a filename or may be a variable that will evaluate into a filename. This will be used in a list of filenames or a list of experiment names iterated through a for loop. >>> for name in (['file1','file2','file3']): .... expClose name .... expClose: ['file1'] expClose: ['file2'] expClose: ['file3'] But when the user tries to use a formal name that is not through a python variable he/she hits an undefine name error: >>> expClose file4 Traceback (most recent call last): File "<console>", line 1, in ? NameError: name 'file4' is not defined If I prescan the <filename> and pack it with quotes I lose the python evaluation of the name. The same problem will occur with all my other option arguments for which there are many. The solution it would seem would be to do the evaluation later within the called function. That way I could assume that all failed eval()'ed names are literals meant for my command and not a python variable/name. If this makes sense, the problem I need to solve is how to deliver the correct namespace dictionary to the called function so I can invoke eval with it. Does this make sense? Regards, Jack |
|
|
|
|
|||
|
|||
| Jack Carter |
|
Alex Martelli
Guest
Posts: n/a
|
Jack Carter <> wrote:
... > The solution it would seem would be to do the evaluation > later within the called function. That way I could assume > that all failed eval()'ed names are literals meant for my > command and not a python variable/name. > > If this makes sense, the problem I need to solve is how to > deliver the correct namespace dictionary to the called function > so I can invoke eval with it. > > Does this make sense? Not very, but then I didn't follow the previous LONG posts on this thread, so I'll just answer this specific question and hope it helps. I'll assume the known variable-names are in some dictionary (such as a locals() or globals() or vars(something)): class WeirdNamespace: def __init__(self, d): self.d = d def __getitem__(self, n): return self.d.get(n,repr(n)) voila: if n is a key in dict d, this returns the corresponding value, otherwise it returns n suitably quoted. Just pass to eval a WeirdNamespace(d) rather than the bare d. It appears to me your user interface is courting trouble: if I mispell 'variablename' as 'varaiblename' I end up creating a file I didn't mean to rather than getting a clean error about unknown variable names. But I'll assume you know your users better than I do and that they _do_ really desire with all their hearts this unholy confusion between variables and constants... Alex |
|
|
|
|
|||
|
|||
| Alex Martelli |
|
Jack Carter
Guest
Posts: n/a
|
On Sep 22, 7:18pm, Alex Martelli wrote:
> Subject: Re: namespace/dictionary quandry > Jack Carter <> wrote: > ... > > The solution it would seem would be to do the evaluation > > later within the called function. That way I could assume > > that all failed eval()'ed names are literals meant for my > > command and not a python variable/name. > > > > If this makes sense, the problem I need to solve is how to > > deliver the correct namespace dictionary to the called function > > so I can invoke eval with it. > > > > Does this make sense? > > Not very, but then I didn't follow the previous LONG posts on this Alex, The length was really taken up with a testcase to make the problem less hand wavey. > thread, so I'll just answer this specific question and hope it helps. > I'll assume the known variable-names are in some dictionary (such as a > locals() or globals() or vars(something)): > > class WeirdNamespace: > def __init__(self, d): self.d = d > def __getitem__(self, n): return self.d.get(n,repr(n)) So, how and or where does this fit in with my example? Does both the call to the function where I want to do the eval() and self.push(line) command have to be in the same namespace and or file for this to work? > > voila: if n is a key in dict d, this returns the corresponding value, > otherwise it returns n suitably quoted. Just pass to eval a > WeirdNamespace(d) rather than the bare d. > > It appears to me your user interface is courting trouble: if I mispell > 'variablename' as 'varaiblename' I end up creating a file I didn't mean > to rather than getting a clean error about unknown variable names. But > I'll assume you know your users better than I do and that they _do_ > really desire with all their hearts this unholy confusion between > variables and constants... This is driven by customers that who want both a type-in command language and a variant using all the power of python for test scripting purposes. If you misspell something and there is the possibility of it screwing something up that we can't tell either in the parser or downstream in the backend of the tool, well that's life in the big city. Thanks, Jack |
|
|
|
|
|||
|
|||
| Jack Carter |
|
Peter Otten
Guest
Posts: n/a
|
Jack Carter wrote:
>> class WeirdNamespace: >> def __init__(self, d): self.d = d >> def __getitem__(self, n): return self.d.get(n,repr(n)) > > So, how and or where does this fit in with my example? > Does both the call to the function where I want to do > the eval() and self.push(line) command have to be in the > same namespace and or file for this to work? I use a slight variation of Alex' suggestion: class Locals(dict): def __getitem__(self, name): try: return dict.__getitem__(self, name) except KeyError: return name class CLI(code.InteractiveConsole): """Simple test of a Python interpreter augmented with custom commands.""" commands = { \ "attach" : "DoAttach" } def __init__(self, locals = None): # Call super-class initializer code.InteractiveConsole.__init__(self, locals, "<console>") self.locals = Locals(self.locals) # Compile regular expression for finding commmands self.regexp = re.compile('[a-z]*') [The rest of the code is the same as in my previous post] Now try it: >>> for name in "abc": fed to the snake: for name in "abc": .... attach name fed to the snake: myparse.DoAttach([name]) .... attach noname fed to the snake: myparse.DoAttach([noname]) .... fed to the snake: DoAttach: ['a'] DoAttach: ['noname'] DoAttach: ['b'] DoAttach: ['noname'] DoAttach: ['c'] DoAttach: ['noname'] Unfortunately this will only work with Python 2.4. Here's a solution that might work for 2.3: [Does not require the above modifications] def process(self, line): indent = line.lstrip() # Attempt to match line against our command regular expression temp_line = string.lstrip(line) len_1 = len(line) len_2 = len(temp_line) white_spaces = len_1-len_2 if white_spaces: front_padding = line[0:white_spaces] match = self.regexp.match(temp_line) if match is not None: #self.write("process 1\n") # Extract the command and argument strings cmd_string = match.group() arg_string = string.lstrip(temp_line[match.end():]) # Find the function for this command in the command dictionary function = self.commands.get(cmd_string) if function is not None: args = parseArgs(arg_string) for arg in args: if arg not in self.locals: self.locals[arg] = arg line = makePythonCall("myparse." + function, args) if white_spaces: line = front_padding + line print "fed to the snake:", line # Return the line to be processed by Python return line I just ensure that all arguments not already in self.locals are added with their name as their value. (I also renamed 'parent' to 'self' - I could not stand it any longer Note that I don't particularly like both hacks and would rather use plain old python functions with standard python syntax instead of your custom language. Peter |
|
|
|
|
|||
|
|||
| Peter Otten |
|
Alex Martelli
Guest
Posts: n/a
|
Jack Carter <> wrote:
... > > > later within the called function. That way I could assume > > > that all failed eval()'ed names are literals meant for my > > > command and not a python variable/name. ... > The length was really taken up with a testcase to make > the problem less hand wavey. If you can't make a shorter example, I'm not gonna make time to study that one, sorry. > > I'll assume the known variable-names are in some dictionary (such as a > > locals() or globals() or vars(something)): > > > > class WeirdNamespace: > > def __init__(self, d): self.d = d > > def __getitem__(self, n): return self.d.get(n,repr(n)) > > So, how and or where does this fit in with my example? > Does both the call to the function where I want to do > the eval() and self.push(line) command have to be in the > same namespace and or file for this to work? All it does is implement what you asked: make a mapping that assumes all unknown variable names are to be taken as literals, starting from a mapping of all 'known variable names' to their values. > > I'll assume you know your users better than I do and that they _do_ > > really desire with all their hearts this unholy confusion between > > variables and constants... > > This is driven by customers that who want both a type-in command > language and a variant using all the power of python for test > scripting purposes. If you misspell something and there is the > possibility of it screwing something up that we can't tell either > in the parser or downstream in the backend of the tool, well that's > life in the big city. I suspect the first time your users want to use a file named 'for' they'll scream bloody murder, but hopefully that won't happen before they've paid you enough for you to retire in comfort to a nice Carribbean island without extradition treaties, so, cheer up. Alex |
|
|
|
|
|||
|
|||
| Alex Martelli |
|
Jack Carter
Guest
Posts: n/a
|
On Sep 22, 8:48pm, Peter Otten wrote:
> Subject: Re: namespace/dictionary quandry > I just ensure that all arguments not already in self.locals are added with > their name as their value. (I also renamed 'parent' to 'self' - I could not > stand it any longer Novice error on my part. Thanks for the correction. > > Note that I don't particularly like both hacks and would rather use plain > old python functions with standard python syntax instead of your custom > language. It makes the programmers job easier, but not the customer who may just want a non-gui debugger commandline session tool. That customer doesn't want to know anything about python or function calls even though underneath the covers that's what they are getting. At the same time, the testers want the full scripting power of python. To make life even more fun, there is a C++ front end and C++ back end that deal with many processes over many nodes. Some of the commands are syncronous and return a value that feed into the python name space and others are asyncronous that may implode somewhere in the future and need to bring the whole mess to some sane state to handle the problem I guess it is my own private hell to work on. Your change seems to work great! Thanks, Jack > > > Peter > |
|
|
|
|
|||
|
|||
| Jack Carter |
|
|
|
| |
![]() |
| Thread Tools | |
|
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| design quandry .. | forums_mp@hotmail.com | C++ | 2 | 10-27-2005 12:01 PM |
| Quandry with the following C code (Intermediate) | BMarsh | C Programming | 7 | 01-14-2005 03:30 PM |
| Client-Side Object Reference Quandry | Fred | ASP .Net | 3 | 07-12-2004 10:28 PM |
| Quandry over which software to use | Phil | Digital Photography | 5 | 02-22-2004 01:40 AM |
| CD quandry | MB | Computer Support | 3 | 09-13-2003 02:14 AM |
Powered by vBulletin®. Copyright ©2000 - 2013, vBulletin Solutions, Inc..
SEO by vBSEO ©2010, Crawlability, Inc. |




