Quantcast

Help on complex nested SEARCH

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Help on complex nested SEARCH

Raymond Lutz
Hi Folks:

I hope this question is not out of place for this list but I've been
banging my head against the proverbial wall for a few hours on this and
I don't want to keep all the fun to myself.;)

I am working on an election integrity project, and I am trying to total
up the votes in sample precincts for one race. I'm trying to set this up
so we can replicate this for many races, with each race topic being
something like "Race005", "Race010" etc.

Here is the SEARCH I have now, and it is pretty close, but I can't seem
to understand how to escape all the $dollars, etc.

|  *Seq*  |  %SEARCH{"Race005_" topic="PrecinctDataForm" type="regex"
multiple="on" nosearch="on" nototal="on"  separator="" format="  
*$pattern(^\| *([^\s]+) *.*)*  |"}%
%SEARCH{"form.name~'*PrecinctDataForm'" type="query" nonoise="on"
noheader="on"
     excludetopic="PrecinctTopicTemplate"
     format ="|  $formfield(SeqNum)  |  $percntSEARCH{ \"Race005_\"
topic=\"PrecinctDataForm\" type=\"regex\" multiple=\"on\"
nosearch=\"on\" nototal=\"on\"  separator=\"\"  format=\"  
$dollar$dollarnop()formfield( $dollarpattern(^\| *([^\s]+) *.*) )  |\"
}$nop%"}%
|  *%CALC{"$ROW(-2)"}%*  \
|  %SEARCH{"Race005_" topic="PrecinctDataForm" type="regex"
multiple="on" nosearch="on" nototal="on"  separator="" format="  
*$percntCALC{\"$SUM( $ABOVE() )\"}$percnt*  |"}%

You can look at the page where I am trying to do this here:
http://www.copswiki.org/w/bin/view/Cops/SearchOfPrecinctFormTest

(They're public pages but if you want to edit, I have to add you to our
ApprovedUsersGroup, which is our way of avoiding evil Google spam
robots. Just let me know and I'll set you up.)

Some brief notes:
1. The PrecinctDataForm is quite large in size, having several hundred
fields. We want to pull out just a few of those fields across all the
precincts (there are 85 precincts in the 5% random sample). This is the
function of the first SEARCH which is working correctly. In this case, I
am pulling out just the presidential race for the November 4 election
(USA), which starts with "Race005_"
2. The function of the second search is to pull out precinct records. At
this point, we are pulling all of them.
3. The embedded SEARCH is to create $formfield(xxxxx) for each of the
selected members of the race (similar to the header).
7. The last SEARCH is to create a line that will sum up the votes
gathered for each precinct.

At present, I can get the $formfield(blahblah) to come out, but I'm at a
loss to figure out the mystery of how to escape this thing, and the art
of escapement is something only briefly described in the documentation
(and usually I sit for hours trying to figure out the solution to the
mystery, which points out that it would be great if there were a way to
provide partial results of SEARCH resolution.)

We wind up with 87 searches for each page, and the sad part is that 85
of those searches produce an identical string that is used just to
format the outer SEARCH. Perhaps there is a way to set a local variable
so the 85 searches can boil down to only 1, and we would have 3 searches
per result. I've never had luck with the local variable concept, but if
there is a puzzle cracker out there, I'd appreciate it. (I just had a
thought that perhaps there is a way to fold this around the other way
and make the formatting search happen first follows by the search for
the records??)

It will not be a good solution to do very much that is custom for each
race. I want to eliminate the literal "Race005" for example so the
search can be the same for every race, and then I will probably make it
a defined constant value in the SitePreferences page (Since I've not had
much luck using %INCLUDE{}% for such complexity as it adds one more
level of complexity, needing to specify the %INCLUDINGTOPIC% all the time.)

(BTW, feel free to look over the other topics of this project. We are
trying to catch our elections officials red-handed...)

--Raymond

--
---------------------------------------
Raymond Lutz
Cognisys, Inc.                
1010 Old Chase Ave., Bldg B            
El Cajon (San Diego Cty), CA 92020 USA
Voice 619-447-3246
http//www.cognisys.com


------------------------------------------------------------------------------
This SF.net email is sponsored by:
High Quality Requirements in a Collaborative Environment.
Download a free trial of Rational Requirements Composer Now!
http://p.sf.net/sfu/www-ibm-com
_______________________________________________
Foswiki-discuss mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/foswiki-discuss
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Help on complex nested SEARCH

lars.eik@gmail.com
Not 100% sure but try replacing $dollar$dollarnop()formfield( $dollarpattern.... with $dollarformfield(....  I have a nested search and use $dollarformfield( and it works.

When testing the nop() thing to stop trying to render wiki-words, not getting ? after words, what worked was %NOP%$dollarformfield(blahbla in the nested section. It don't look like intended that way from the formatted search doc, it says $nop is good for nested searches.  Neither $nop nor $dollarnop worked for me.

Lars

On Thu, Apr 9, 2009 at 8:01 AM, Raymond Lutz <[hidden email]> wrote:
Hi Folks:

I hope this question is not out of place for this list but I've been
banging my head against the proverbial wall for a few hours on this and
I don't want to keep all the fun to myself.;)

I am working on an election integrity project, and I am trying to total
up the votes in sample precincts for one race. I'm trying to set this up
so we can replicate this for many races, with each race topic being
something like "Race005", "Race010" etc.

Here is the SEARCH I have now, and it is pretty close, but I can't seem
to understand how to escape all the $dollars, etc.

|  *Seq*  |  %SEARCH{"Race005_" topic="PrecinctDataForm" type="regex"
multiple="on" nosearch="on" nototal="on"  separator="" format="
*$pattern(^\| *([^\s]+) *.*)*  |"}%
%SEARCH{"form.name~'*PrecinctDataForm'" type="query" nonoise="on"
noheader="on"
    excludetopic="PrecinctTopicTemplate"
    format ="|  $formfield(SeqNum)  |  $percntSEARCH{ \"Race005_\"
topic=\"PrecinctDataForm\" type=\"regex\" multiple=\"on\"
nosearch=\"on\" nototal=\"on\"  separator=\"\"  format=\"
$dollar$dollarnop()formfield( $dollarpattern(^\| *([^\s]+) *.*) )  |\"
}$nop%"}%
|  *%CALC{"$ROW(-2)"}%*  \
|  %SEARCH{"Race005_" topic="PrecinctDataForm" type="regex"
multiple="on" nosearch="on" nototal="on"  separator="" format="
*$percntCALC{\"$SUM( $ABOVE() )\"}$percnt*  |"}%

You can look at the page where I am trying to do this here:
http://www.copswiki.org/w/bin/view/Cops/SearchOfPrecinctFormTest

(They're public pages but if you want to edit, I have to add you to our
ApprovedUsersGroup, which is our way of avoiding evil Google spam
robots. Just let me know and I'll set you up.)

Some brief notes:
1. The PrecinctDataForm is quite large in size, having several hundred
fields. We want to pull out just a few of those fields across all the
precincts (there are 85 precincts in the 5% random sample). This is the
function of the first SEARCH which is working correctly. In this case, I
am pulling out just the presidential race for the November 4 election
(USA), which starts with "Race005_"
2. The function of the second search is to pull out precinct records. At
this point, we are pulling all of them.
3. The embedded SEARCH is to create $formfield(xxxxx) for each of the
selected members of the race (similar to the header).
7. The last SEARCH is to create a line that will sum up the votes
gathered for each precinct.

At present, I can get the $formfield(blahblah) to come out, but I'm at a
loss to figure out the mystery of how to escape this thing, and the art
of escapement is something only briefly described in the documentation
(and usually I sit for hours trying to figure out the solution to the
mystery, which points out that it would be great if there were a way to
provide partial results of SEARCH resolution.)

We wind up with 87 searches for each page, and the sad part is that 85
of those searches produce an identical string that is used just to
format the outer SEARCH. Perhaps there is a way to set a local variable
so the 85 searches can boil down to only 1, and we would have 3 searches
per result. I've never had luck with the local variable concept, but if
there is a puzzle cracker out there, I'd appreciate it. (I just had a
thought that perhaps there is a way to fold this around the other way
and make the formatting search happen first follows by the search for
the records??)

It will not be a good solution to do very much that is custom for each
race. I want to eliminate the literal "Race005" for example so the
search can be the same for every race, and then I will probably make it
a defined constant value in the SitePreferences page (Since I've not had
much luck using %INCLUDE{}% for such complexity as it adds one more
level of complexity, needing to specify the %INCLUDINGTOPIC% all the time.)

(BTW, feel free to look over the other topics of this project. We are
trying to catch our elections officials red-handed...)

--Raymond

--
---------------------------------------
Raymond Lutz
Cognisys, Inc.
1010 Old Chase Ave., Bldg B
El Cajon (San Diego Cty), CA 92020 USA
Voice 619-447-3246
http//www.cognisys.com


------------------------------------------------------------------------------
This SF.net email is sponsored by:
High Quality Requirements in a Collaborative Environment.
Download a free trial of Rational Requirements Composer Now!
http://p.sf.net/sfu/www-ibm-com
_______________________________________________
Foswiki-discuss mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/foswiki-discuss


------------------------------------------------------------------------------
This SF.net email is sponsored by:
High Quality Requirements in a Collaborative Environment.
Download a free trial of Rational Requirements Composer Now!
http://p.sf.net/sfu/www-ibm-com
_______________________________________________
Foswiki-discuss mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/foswiki-discuss
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Help on complex nested SEARCH

Raymond Lutz
Thanks Lars, you are logically correct, I think, but that does not work for other reasons that I am only (after sleeping on this problem) now starting to probably understand.  When you do what you said, we get the string ".) )" in each cell. I can tell you what is happening but I still don't know what a good solution is.

First, let me show the search that generates the desired cell contents:
|  %SEARCH{ "Race005_" topic="PrecinctDataForm" type="regex" multiple="on" nosearch="on" nototal="on"  separator=""  format="  $dollarformfield( $pattern(^\| *([^\s]+) *.*) )  |" }%

This results in (effectively):
|  $formfield( Race005_MCCAIN_PALIN )  |  $formfield( Race005_MCKINNEY_CLEMENTE )  | ... etc.
just like we would expect (and what is needed in the other format string). Note that $dollarformfield is required so it does not evaluate the $formfield right away. I added this to the page below at the top.
http://www.copswiki.org/w/bin/view/Cops/SearchOfPrecinctFormTest

Inflating that with escapes (% --> $percnt; " --> \"; $ --> $dollar) I end up with
|  $percntSEARCH{ \"Race005_\" topic=\"PrecinctDataForm\" type=\"regex\" multiple=\"on\" nosearch=\"on\" nototal=\"on\"  separator=\"\"  format=\"   $dollardollarformfield( $dollarpattern(^\| *([^\s]+) *.*) )  |\" }$percnt
which should fit into the format quotes and survive.

Indeed, it does, and we get exactly that:
0015 $formfield( Race005_MCCAIN_PALIN ) $formfield( Race005_MCKINNEY_CLEMENTE ) $formfield( Race005_KEYES_DRAKE ) $formfield( Race005_OBAMA_BIDEN ) $formfield( Race005_BARR_ROOT ) $formfield( Race005_NADER_GONZALEZ )

i.e. the literal $formfield( Race005_names.. ) is found in each cell of the table. The problem is that the outer search as already completed, and so inserting $formfield(blahblah) into the cell of the table does nothing.

This may be impossible, given the fact that it is not possible to "hold off" an outer search to wait for an inner search to complete. What we would like to do is to have the |  $formfield(blah)  |  $formfield(... | part formed FIRST, and then have the outer search use that result as an ACTIVE PART OF THE format string. Instead, by escaping the inner SEARCH, we are causing it to complete AFTER the table is created, thereby causing the $formfield() syntax to be useless.

If it were possible, I would like to do this:
   * Set LOCALVARIABLE = %SEARCH{ "Race005_" topic="PrecinctDataForm" type="regex" multiple="on" nosearch="on" nototal="on"  separator=""  format="  $dollarformfield( $pattern(^\| *([^\s]+) *.*) )  |" }%

And then the table for one race would be:
|  *Seq*  |  %SEARCH{"Race005_" topic="PrecinctDataForm" type="regex" multiple="on" nosearch="on" nototal="on"  separator="" format="  *$pattern(^\| *([^\s]+) *.*)*  |"}%
%SEARCH{"form.name~'*PrecinctDataForm'" type="query" nonoise="on" noheader="on"
     excludetopic="PrecinctTopicTemplate" format ="|  [[$topic][$formfield(SeqNum)]]  |  %LOCALVARIABLE%"}%
|  *%CALC{"$ROW(-2)"}%*  \
|  %SEARCH{"Race005_" topic="PrecinctDataForm" type="regex" multiple="on" nosearch="on" nototal="on"  separator="" format="  *$percntCALC{\"$SUM( $ABOVE() )\"}$percnt*  |"}%

As long as it were the case that the LOCALVARIABLE were actually evaluated prior to the search. Note that once I figure this out, I want to replace "Race005" in the current example with %TOPIC%:
   * Set LOCALVARIABLE = %SEARCH{ "%TOPIC%_" topic="PrecinctDataForm" type="regex" multiple="on" nosearch="on" nototal="on"  separator=""  format="  $dollarformfield( $pattern(^\| *([^\s]+) *.*) )  |" }%
I.e. based on the topic name for that race (so a SitePreferences "Set" won't work). [I need to make up many topics, one per race.]

>From a syntax standpoint (and not implementation) I would LOVE the (LOGICAL!) following syntax to work:

|  *Seq*  |  %SEARCH{"Race005_" topic="PrecinctDataForm" type="regex" multiple="on" nosearch="on" nototal="on"  separator="" format="  *$pattern(^\| *([^\s]+) *.*)*  |"}%
$percntSEARCH{"form.name~'*PrecinctDataForm'" type="query" nonoise="on" noheader="on"
     excludetopic="PrecinctTopicTemplate"
     format ="|  [[$topic][$formfield(SeqNum)]]  |  %SEARCH{ "Race005_" topic="PrecinctDataForm" type="regex" multiple="on" nosearch="on" nototal="on"  separator=""  format="   $dollarformfield( $pattern(^\| *([^\s]+) *.*) )  |" }%"}$percnt
|  *%CALC{"$ROW(-2)"}%*  \
|  %SEARCH{"Race005_" topic="PrecinctDataForm" type="regex" multiple="on" nosearch="on" nototal="on"  separator="" format="  *$percntCALC{\"$SUM( $ABOVE() )\"}$percnt*  |"}%

But indeed, the $percnt syntax does not force any reevaluation of the page, and you get this (See bottom example on link above).
$percntSEARCH{"form.name~'*PrecinctDataForm'" type="query" nonoise="on" noheader="on" excludetopic="PrecinctTopicTemplate" format ="| $formfield(SeqNum) | $formfield( Race005_MCCAIN_PALIN ) | $formfield( Race005_MCKINNEY_CLEMENTE ) | $formfield( Race005_KEYES_DRAKE ) | $formfield( Race005_OBAMA_BIDEN ) | $formfield( Race005_BARR_ROOT ) | $formfield( Race005_NADER_GONZALEZ ) |"}$percnt

I think a solution lies in one of two directions:
1. the local variable idea, perhaps with CALC.
2. or somehow embed the outer search in some other syntax to force it to be evaluated again.

Ideas?

Lars Eik wrote:
Not 100% sure but try replacing $dollar$dollarnop()formfield( $dollarpattern.... with $dollarformfield(....  I have a nested search and use $dollarformfield( and it works.

When testing the nop() thing to stop trying to render wiki-words, not getting ? after words, what worked was %NOP%$dollarformfield(blahbla in the nested section. It don't look like intended that way from the formatted search doc, it says $nop is good for nested searches.  Neither $nop nor $dollarnop worked for me.

Lars

On Thu, Apr 9, 2009 at 8:01 AM, Raymond Lutz <[hidden email]> wrote:
Hi Folks:

I hope this question is not out of place for this list but I've been
banging my head against the proverbial wall for a few hours on this and
I don't want to keep all the fun to myself.;)

I am working on an election integrity project, and I am trying to total
up the votes in sample precincts for one race. I'm trying to set this up
so we can replicate this for many races, with each race topic being
something like "Race005", "Race010" etc.

Here is the SEARCH I have now, and it is pretty close, but I can't seem
to understand how to escape all the $dollars, etc.

|  *Seq*  |  %SEARCH{"Race005_" topic="PrecinctDataForm" type="regex"
multiple="on" nosearch="on" nototal="on"  separator="" format="
*$pattern(^\| *([^\s]+) *.*)*  |"}%
%SEARCH{"form.name~'*PrecinctDataForm'" type="query" nonoise="on"
noheader="on"
    excludetopic="PrecinctTopicTemplate"
    format ="|  $formfield(SeqNum)  |  $percntSEARCH{ \"Race005_\"
topic=\"PrecinctDataForm\" type=\"regex\" multiple=\"on\"
nosearch=\"on\" nototal=\"on\"  separator=\"\"  format=\"
$dollar$dollarnop()formfield( $dollarpattern(^\| *([^\s]+) *.*) )  |\"
}$nop%"}%
|  *%CALC{"$ROW(-2)"}%*  \
|  %SEARCH{"Race005_" topic="PrecinctDataForm" type="regex"
multiple="on" nosearch="on" nototal="on"  separator="" format="
*$percntCALC{\"$SUM( $ABOVE() )\"}$percnt*  |"}%

You can look at the page where I am trying to do this here:
http://www.copswiki.org/w/bin/view/Cops/SearchOfPrecinctFormTest

(They're public pages but if you want to edit, I have to add you to our
ApprovedUsersGroup, which is our way of avoiding evil Google spam
robots. Just let me know and I'll set you up.)

Some brief notes:
1. The PrecinctDataForm is quite large in size, having several hundred
fields. We want to pull out just a few of those fields across all the
precincts (there are 85 precincts in the 5% random sample). This is the
function of the first SEARCH which is working correctly. In this case, I
am pulling out just the presidential race for the November 4 election
(USA), which starts with "Race005_"
2. The function of the second search is to pull out precinct records. At
this point, we are pulling all of them.
3. The embedded SEARCH is to create $formfield(xxxxx) for each of the
selected members of the race (similar to the header).
7. The last SEARCH is to create a line that will sum up the votes
gathered for each precinct.

At present, I can get the $formfield(blahblah) to come out, but I'm at a
loss to figure out the mystery of how to escape this thing, and the art
of escapement is something only briefly described in the documentation
(and usually I sit for hours trying to figure out the solution to the
mystery, which points out that it would be great if there were a way to
provide partial results of SEARCH resolution.)

We wind up with 87 searches for each page, and the sad part is that 85
of those searches produce an identical string that is used just to
format the outer SEARCH. Perhaps there is a way to set a local variable
so the 85 searches can boil down to only 1, and we would have 3 searches
per result. I've never had luck with the local variable concept, but if
there is a puzzle cracker out there, I'd appreciate it. (I just had a
thought that perhaps there is a way to fold this around the other way
and make the formatting search happen first follows by the search for
the records??)

It will not be a good solution to do very much that is custom for each
race. I want to eliminate the literal "Race005" for example so the
search can be the same for every race, and then I will probably make it
a defined constant value in the SitePreferences page (Since I've not had
much luck using %INCLUDE{}% for such complexity as it adds one more
level of complexity, needing to specify the %INCLUDINGTOPIC% all the time.)

(BTW, feel free to look over the other topics of this project. We are
trying to catch our elections officials red-handed...)

--Raymond

--
---------------------------------------
Raymond Lutz
Cognisys, Inc.
1010 Old Chase Ave., Bldg B
El Cajon (San Diego Cty), CA 92020 USA
Voice 619-447-3246
http//www.cognisys.com


------------------------------------------------------------------------------
This SF.net email is sponsored by:
High Quality Requirements in a Collaborative Environment.
Download a free trial of Rational Requirements Composer Now!
http://p.sf.net/sfu/www-ibm-com
_______________________________________________
Foswiki-discuss mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/foswiki-discuss


------------------------------------------------------------------------------ This SF.net email is sponsored by: High Quality Requirements in a Collaborative Environment. Download a free trial of Rational Requirements Composer Now! http://p.sf.net/sfu/www-ibm-com

_______________________________________________ Foswiki-discuss mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/foswiki-discuss

-- 
---------------------------------------
Raymond Lutz
Cognisys, Inc.                
1010 Old Chase Ave., Bldg B             
El Cajon (San Diego Cty), CA 92020 USA
Voice 619-447-3246
http//www.cognisys.com

------------------------------------------------------------------------------
This SF.net email is sponsored by:
High Quality Requirements in a Collaborative Environment.
Download a free trial of Rational Requirements Composer Now!
http://p.sf.net/sfu/www-ibm-com
_______________________________________________
Foswiki-discuss mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/foswiki-discuss
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Help on complex nested SEARCH

lars.eik@gmail.com
Raymond,

Did you try this? in the nested search:  $formfield( $dollarpattern(....

Don't escape the $formfield because you want it to render the formfield of the outer search but with the fieldname of the nested search's pattern match. As long as you see that $dollarpattern gives the fieldnames it should work.

The nested search could never find a formfield within the nested search (escaped that is) because you search the PrecinctForm topic, which do not have the form attached and therefore cannot give back anything other than your text: "$formfield( Race005_MCKINNEY_CLEMENTE )".

Lars


On Thu, Apr 9, 2009 at 6:08 PM, Raymond Lutz <[hidden email]> wrote:
Thanks Lars, you are logically correct, I think, but that does not work for other reasons that I am only (after sleeping on this problem) now starting to probably understand.  When you do what you said, we get the string ".) )" in each cell. I can tell you what is happening but I still don't know what a good solution is.

First, let me show the search that generates the desired cell contents:
|  %SEARCH{ "Race005_" topic="PrecinctDataForm" type="regex" multiple="on" nosearch="on" nototal="on"  separator=""  format="  $dollarformfield( $pattern(^\| *([^\s]+) *.*) )  |" }%

This results in (effectively):
|  $formfield( Race005_MCCAIN_PALIN )  |  $formfield( Race005_MCKINNEY_CLEMENTE )  | ... etc.
just like we would expect (and what is needed in the other format string). Note that $dollarformfield is required so it does not evaluate the $formfield right away. I added this to the page below at the top. Inflating that with escapes (% --> $percnt; " --> \"; $ --> $dollar) I end up with
|  $percntSEARCH{ \"Race005_\" topic=\"PrecinctDataForm\" type=\"regex\" multiple=\"on\" nosearch=\"on\" nototal=\"on\"  separator=\"\"  format=\"   $dollardollarformfield( $dollarpattern(^\| *([^\s]+) *.*) )  |\" }$percnt
which should fit into the format quotes and survive.

Indeed, it does, and we get exactly that:
0015 $formfield( Race005_MCCAIN_PALIN ) $formfield( Race005_MCKINNEY_CLEMENTE ) $formfield( Race005_KEYES_DRAKE ) $formfield( Race005_OBAMA_BIDEN ) $formfield( Race005_BARR_ROOT ) $formfield( Race005_NADER_GONZALEZ )

i.e. the literal $formfield( Race005_names.. ) is found in each cell of the table. The problem is that the outer search as already completed, and so inserting $formfield(blahblah) into the cell of the table does nothing.

This may be impossible, given the fact that it is not possible to "hold off" an outer search to wait for an inner search to complete. What we would like to do is to have the |  $formfield(blah)  |  $formfield(... | part formed FIRST, and then have the outer search use that result as an ACTIVE PART OF THE format string. Instead, by escaping the inner SEARCH, we are causing it to complete AFTER the table is created, thereby causing the $formfield() syntax to be useless.

If it were possible, I would like to do this:
   * Set LOCALVARIABLE = %SEARCH{ "Race005_" topic="PrecinctDataForm" type="regex" multiple="on" nosearch="on" nototal="on"  separator=""  format="  $dollarformfield( $pattern(^\| *([^\s]+) *.*) )  |" }%

And then the table for one race would be:

|  *Seq*  |  %SEARCH{"Race005_" topic="PrecinctDataForm" type="regex" multiple="on" nosearch="on" nototal="on"  separator="" format="  *$pattern(^\| *([^\s]+) *.*)*  |"}%
%SEARCH{"form.name~'*PrecinctDataForm'" type="query" nonoise="on" noheader="on"
     excludetopic="PrecinctTopicTemplate" format ="|  [[$topic][$formfield(SeqNum)]]  |  %LOCALVARIABLE%"}%

|  *%CALC{"$ROW(-2)"}%*  \
|  %SEARCH{"Race005_" topic="PrecinctDataForm" type="regex" multiple="on" nosearch="on" nototal="on"  separator="" format="  *$percntCALC{\"$SUM( $ABOVE() )\"}$percnt*  |"}%

As long as it were the case that the LOCALVARIABLE were actually evaluated prior to the search. Note that once I figure this out, I want to replace "Race005" in the current example with %TOPIC%:
   * Set LOCALVARIABLE = %SEARCH{ "%TOPIC%_" topic="PrecinctDataForm" type="regex" multiple="on" nosearch="on" nototal="on"  separator=""  format="  $dollarformfield( $pattern(^\| *([^\s]+) *.*) )  |" }%
I.e. based on the topic name for that race (so a SitePreferences "Set" won't work). [I need to make up many topics, one per race.]

>From a syntax standpoint (and not implementation) I would LOVE the (LOGICAL!) following syntax to work:


|  *Seq*  |  %SEARCH{"Race005_" topic="PrecinctDataForm" type="regex" multiple="on" nosearch="on" nototal="on"  separator="" format="  *$pattern(^\| *([^\s]+) *.*)*  |"}%
$percntSEARCH{"form.name~'*PrecinctDataForm'" type="query" nonoise="on" noheader="on"
     excludetopic="PrecinctTopicTemplate"
     format ="|  [[$topic][$formfield(SeqNum)]]  |  %SEARCH{ "Race005_" topic="PrecinctDataForm" type="regex" multiple="on" nosearch="on" nototal="on"  separator=""  format="   $dollarformfield( $pattern(^\| *([^\s]+) *.*) )  |" }%"}$percnt

|  *%CALC{"$ROW(-2)"}%*  \
|  %SEARCH{"Race005_" topic="PrecinctDataForm" type="regex" multiple="on" nosearch="on" nototal="on"  separator="" format="  *$percntCALC{\"$SUM( $ABOVE() )\"}$percnt*  |"}%

But indeed, the $percnt syntax does not force any reevaluation of the page, and you get this (See bottom example on link above).
$percntSEARCH{"form.name~'*PrecinctDataForm'" type="query" nonoise="on" noheader="on" excludetopic="PrecinctTopicTemplate" format ="| $formfield(SeqNum) | $formfield( Race005_MCCAIN_PALIN ) | $formfield( Race005_MCKINNEY_CLEMENTE ) | $formfield( Race005_KEYES_DRAKE ) | $formfield( Race005_OBAMA_BIDEN ) | $formfield( Race005_BARR_ROOT ) | $formfield( Race005_NADER_GONZALEZ ) |"}$percnt

I think a solution lies in one of two directions:
1. the local variable idea, perhaps with CALC.
2. or somehow embed the outer search in some other syntax to force it to be evaluated again.

Ideas?

Lars Eik wrote:
Not 100% sure but try replacing $dollar$dollarnop()formfield( $dollarpattern.... with $dollarformfield(....  I have a nested search and use $dollarformfield( and it works.

When testing the nop() thing to stop trying to render wiki-words, not getting ? after words, what worked was %NOP%$dollarformfield(blahbla in the nested section. It don't look like intended that way from the formatted search doc, it says $nop is good for nested searches.  Neither $nop nor $dollarnop worked for me.

Lars

On Thu, Apr 9, 2009 at 8:01 AM, Raymond Lutz <[hidden email]> wrote:
Hi Folks:

I hope this question is not out of place for this list but I've been
banging my head against the proverbial wall for a few hours on this and
I don't want to keep all the fun to myself.;)

I am working on an election integrity project, and I am trying to total
up the votes in sample precincts for one race. I'm trying to set this up
so we can replicate this for many races, with each race topic being
something like "Race005", "Race010" etc.

Here is the SEARCH I have now, and it is pretty close, but I can't seem
to understand how to escape all the $dollars, etc.

|  *Seq*  |  %SEARCH{"Race005_" topic="PrecinctDataForm" type="regex"
multiple="on" nosearch="on" nototal="on"  separator="" format="
*$pattern(^\| *([^\s]+) *.*)*  |"}%
%SEARCH{"form.name~'*PrecinctDataForm'" type="query" nonoise="on"
noheader="on"
    excludetopic="PrecinctTopicTemplate"
    format ="|  $formfield(SeqNum)  |  $percntSEARCH{ \"Race005_\"
topic=\"PrecinctDataForm\" type=\"regex\" multiple=\"on\"
nosearch=\"on\" nototal=\"on\"  separator=\"\"  format=\"
$dollar$dollarnop()formfield( $dollarpattern(^\| *([^\s]+) *.*) )  |\"
}$nop%"}%
|  *%CALC{"$ROW(-2)"}%*  \
|  %SEARCH{"Race005_" topic="PrecinctDataForm" type="regex"
multiple="on" nosearch="on" nototal="on"  separator="" format="
*$percntCALC{\"$SUM( $ABOVE() )\"}$percnt*  |"}%

You can look at the page where I am trying to do this here:
http://www.copswiki.org/w/bin/view/Cops/SearchOfPrecinctFormTest

(They're public pages but if you want to edit, I have to add you to our
ApprovedUsersGroup, which is our way of avoiding evil Google spam
robots. Just let me know and I'll set you up.)

Some brief notes:
1. The PrecinctDataForm is quite large in size, having several hundred
fields. We want to pull out just a few of those fields across all the
precincts (there are 85 precincts in the 5% random sample). This is the
function of the first SEARCH which is working correctly. In this case, I
am pulling out just the presidential race for the November 4 election
(USA), which starts with "Race005_"
2. The function of the second search is to pull out precinct records. At
this point, we are pulling all of them.
3. The embedded SEARCH is to create $formfield(xxxxx) for each of the
selected members of the race (similar to the header).
7. The last SEARCH is to create a line that will sum up the votes
gathered for each precinct.

At present, I can get the $formfield(blahblah) to come out, but I'm at a
loss to figure out the mystery of how to escape this thing, and the art
of escapement is something only briefly described in the documentation
(and usually I sit for hours trying to figure out the solution to the
mystery, which points out that it would be great if there were a way to
provide partial results of SEARCH resolution.)

We wind up with 87 searches for each page, and the sad part is that 85
of those searches produce an identical string that is used just to
format the outer SEARCH. Perhaps there is a way to set a local variable
so the 85 searches can boil down to only 1, and we would have 3 searches
per result. I've never had luck with the local variable concept, but if
there is a puzzle cracker out there, I'd appreciate it. (I just had a
thought that perhaps there is a way to fold this around the other way
and make the formatting search happen first follows by the search for
the records??)

It will not be a good solution to do very much that is custom for each
race. I want to eliminate the literal "Race005" for example so the
search can be the same for every race, and then I will probably make it
a defined constant value in the SitePreferences page (Since I've not had
much luck using %INCLUDE{}% for such complexity as it adds one more
level of complexity, needing to specify the %INCLUDINGTOPIC% all the time.)

(BTW, feel free to look over the other topics of this project. We are
trying to catch our elections officials red-handed...)

--Raymond

--
---------------------------------------
Raymond Lutz
Cognisys, Inc.
1010 Old Chase Ave., Bldg B
El Cajon (San Diego Cty), CA 92020 USA
Voice 619-447-3246
http//www.cognisys.com


------------------------------------------------------------------------------
This SF.net email is sponsored by:
High Quality Requirements in a Collaborative Environment.
Download a free trial of Rational Requirements Composer Now!
http://p.sf.net/sfu/www-ibm-com
_______________________________________________
Foswiki-discuss mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/foswiki-discuss


------------------------------------------------------------------------------ This SF.net email is sponsored by: High Quality Requirements in a Collaborative Environment. Download a free trial of Rational Requirements Composer Now! http://p.sf.net/sfu/www-ibm-com

_______________________________________________ Foswiki-discuss mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/foswiki-discuss

-- 
---------------------------------------
Raymond Lutz
Cognisys, Inc.                
1010 Old Chase Ave., Bldg B             
El Cajon (San Diego Cty), CA 92020 USA
Voice 619-447-3246
http//www.cognisys.com

------------------------------------------------------------------------------
This SF.net email is sponsored by:
High Quality Requirements in a Collaborative Environment.
Download a free trial of Rational Requirements Composer Now!
http://p.sf.net/sfu/www-ibm-com
_______________________________________________
Foswiki-discuss mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/foswiki-discuss



------------------------------------------------------------------------------
This SF.net email is sponsored by:
High Quality Requirements in a Collaborative Environment.
Download a free trial of Rational Requirements Composer Now!
http://p.sf.net/sfu/www-ibm-com
_______________________________________________
Foswiki-discuss mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/foswiki-discuss
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Help on complex nested SEARCH - solved!

Raymond Lutz
Well, I figured this out, and it had nothing to do with the escaping,
but I had to use an INCLUDE to get the format portion to be executed
first. Luckily, INCLUDE is evaluated before the SEARCH that has it in
its format="" clause. ( I thank the list for letting me work on this
with your help...)

For your interest (and to complete discussion of this topic) here is the
solution to the puzzle: I have this SEARCH on the main page for a given
race:
|  *Seq*  |  %SEARCH{"Race005_" topic="PrecinctDataForm" type="regex"
multiple="on" nosearch="on" nototal="on"  separator="" format="  
*$pattern(^\| *([^\s]+) *.*)*  |"}%
%SEARCH{"form.name~'*PrecinctDataForm'" type="query" nonoise="on"
noheader="on"
     excludetopic="PrecinctTopicTemplate"
     format ="|  [[$topic][$formfield(SeqNum)]]  |  
%INCLUDE{"InnerLoopInclude" RACE="Race005"}%"}%
|  *%CALC{"$ROW(-2)"}%*  \
|  %SEARCH{"Race005_" topic="PrecinctDataForm" type="regex"
multiple="on" nosearch="on" nototal="on"  separator="" format="  
*$percntCALC{\"$SUM( $ABOVE() )\"}$percnt*  |"}%

And, the topic InnerLoopInclude has this section to be included:
%SEARCH{ "%RACE%_" topic="PrecinctDataForm" type="regex" multiple="on"
nosearch="on" nototal="on"  separator=""  format="  $dollarformfield(
$pattern(^\| *([^\s]+) *.*) )  |" }%

"It would be nice" if this were documented as a way to delay execution
of the outer SEARCH until the format clause is constructed.

I have this analysis of the situation and suggestion.

The intent is to search the raw PrecinctDataForm topic to find the
(sometimes many) members of a given race, so that those can be pulled
out of all the topics (85) that have that form attached. So, unlike the
normal practice of filling in a result after the outer search is
completed (by embedding the SEARCH in the format string (escaped) ) the
intention here is to build the format string from a subset of the
information in the PrecinctDataForm topic to construct the final SEARCH
which can then be executed.

The way foswiki is implemented, you can't build the format statement
with active elements (like $formfield(blahblah)) that you expect to use
in the outer SEARCH because the outer search executes first, and then
the inner elements just wind up being shown explicitly.

I added your suggestion to the page, and it produced ". ) )" in each cell.

Looking at just the format statement of the outer search:
 format ="|  [[$topic][$formfield(SeqNum)]]  |  $percntSEARCH{
\"Race005_\" topic=\"PrecinctDataForm\" type=\"regex\" multiple=\"on\"
nosearch=\"on\" nototal=\"on\"  separator=\"\"  format=\"   $formfield(
$dollarpattern(^\| *([^\s]+) *.*) )  |\" }$percnt"

after the search is executed, the intermediate result is:
|   [[TopicWikiWord][SeqNumNNNN]]  |  %SEARCH{ "Race005_"
topic="PrecinctDataForm" type="regex" multiple="on" nosearch="on"
nototal="on"  separator=""  format="   $formfield( $dollarpattern(^\|
*([^\s]+) *.*) )  |" }%"
But, it tries to find in the PrecinctDataForm the formfield named
"$dollarpattern(^\| *([^\s]+" because it looks for the first closing
paren. That leaves *.*) ) which looks like a bolded period and two
closing parens.

I thank the list for your discretion in helping me think this through.

--Raymond

--
---------------------------------------
Raymond Lutz
Cognisys, Inc.                
1010 Old Chase Ave., Bldg B            
El Cajon (San Diego Cty), CA 92020 USA
Voice 619-447-3246
http//www.cognisys.com


------------------------------------------------------------------------------
This SF.net email is sponsored by:
High Quality Requirements in a Collaborative Environment.
Download a free trial of Rational Requirements Composer Now!
http://p.sf.net/sfu/www-ibm-com
_______________________________________________
Foswiki-discuss mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/foswiki-discuss
Loading...