WeBWorK Problems

issue with scaffold.pl or contextForm.pl

issue with scaffold.pl or contextForm.pl

by Alex Jordan -
Number of replies: 12

Consider the following MWE

DOCUMENT();
loadMacros(
  "PGstandard.pl",
  "MathObjects.pl",
  "PGML.pl",
  "contextForm.pl",
  "scaffold.pl",
);
Context("Form");
$ans = Formula("x/2");
#Scaffold::Begin();
#Section::Begin("");
BEGIN_PGML
[`\frac{x}{2}=`][_]{$ans}
END_PGML
#Section::End();
#Scaffold::End();
ENDDOCUMENT();

I think it works just fine. You can enter "x/2" and it is correct. Or "2x/4" and get a special feedback message.

Now uncomment the four lines for scaffolding. And then enter "x/2" as an answer. At some level it is assessed as correct because the scaffold styling turns green. However the results table says it is incorrect and there is red styling around the input field. If I peek inside the results hash, the score is 0 (which is probably what leads to the result table saying it is incorrect and the red styling).

If I change the context from Form to Numeric, then the issue is not there. Alternatively, leave it in context Form and change the answer to just be "x". Then there is no problem either. ("x" is completely regarded as correct and "2x/2" gives a special feedback message.)

On the one hand, since this works without scaffolding, I would think something is not handled correctly in scaffold.pl. I looked in there though and could not see what the issue might be. Although a suspect is the code in `sub section_answers`.

On the other hand, I have little confidence that I wrote the answer checker for contextForm.pl in the right way (in the OPL PCC macros folder). Maybe it can be written in some more robust way that will work with scaffold.pl.

Is anyone available to help me figure this out?
In reply to Alex Jordan

Re: issue with scaffold.pl or contextForm.pl

by Alex Jordan -

In scaffold.pl, in the Section package, there is this line:

$PG_ANSWERS_HASH = $Scaffold::PG_ANSWERS_HASH;

When I print $PG_ANSWERS_HASH, there is nothing there. It has no keys. Meanwhile, in the Scaffold package, there is

our $PG_ANSWERS_HASH = $main::PG->{PG_ANSWERS_HASH};

When I print this one, I see what I should see (a hash with one key AnSwEr0001). And this the case even if I take context "Form" out of consideration and run a problem with context "Numeric".  I don't understand enough about what is happening here with perl packages, but it seems wrong that within the Section package, this hash reference is always coming up empty.


Edited to add:

The plot thickens. If I convert my sample exercise to not use PGML, the issue goes away. So PGML is an element to the issue.




In reply to Alex Jordan

Re: issue with scaffold.pl or contextForm.pl

by Glenn Rice -
It depends on where you print $PG_ANSWERS_HASH. If you try to print directly below the definitions for both of them, you will get nothing. The definitions are just taking a reference to the $main::PG->{PG_ANSWERS_HASH} and the $Scaffold::PG_ANSWERS_HASH respectively, and those references are taken at compile time. At that time all of those hashes are empty. The problem text where the answers are located has not been processed yet by PGML. When the problem text is processed by PGML, then the answer labels are added as keys to $main::PG->{PG_ANSWERS_HASH}, and since the others are just references to that, they also get access to those. If you print the contents of those variable where they are used, you will see that they do indeed have contents.
In reply to Glenn Rice

Re: issue with scaffold.pl or contextForm.pl

by Alex Jordan -
I was printing them where used. Did you use my sample problem in a 2.17 server?
In reply to Alex Jordan

Re: issue with scaffold.pl or contextForm.pl

by Alex Jordan -
OK, so if I print it from within the subroutine assigned_ans, I get nothing. If I print from within new_answers, I see it. Does it make sense to see nothing from inside assigned_ans? That subroutine has code that loops through the keys of that hash.
In reply to Alex Jordan

Re: issue with scaffold.pl or contextForm.pl

by Glenn Rice -
Since in this problem there is only one scaffold, if you print the keys in assigned_ans you will see nothing. The only time that is called for this problem is at the beginning of the only section, and at that time PGML still has not started processing the problem text. You will see them in new_answers because that is called at the end of the section, after PGML has processed the problem.
In reply to Glenn Rice

Re: issue with scaffold.pl or contextForm.pl

by Alex Jordan -
OK, thanks. Well that was a distraction them. I still have the original issue undiagnosed. I've learned that it only happens when PGML is used to make the answer blank though.
In reply to Alex Jordan

Re: issue with scaffold.pl or contextForm.pl

by Alex Jordan -
I take back my claim about PGML. I must have been confused about what I was running. Here, even without PGML, is a more minimal version that demonstrates the issue.

```
DOCUMENT();

loadMacros(
  "PGstandard.pl",
  "contextForm.pl",
  "scaffold.pl",
);

Context("Form");
$ans = Formula("x/2");

Scaffold::Begin();
Section::Begin("");

BEGIN_TEXT
\(\frac{x}{2}=\) \{ans_rule()\}
END_TEXT

ANS($ans->cmp);

Section::End();
Scaffold::End();

ENDDOCUMENT();
```


Hmm. If I move the ANS() call to *after* the Scaffold is ended (or even just after the Section is ended), the behavior changes (my original issue goes away). So maybe that is key. Having the ANS() call within the scaffold is like using PGML, and that is likely what confused me earlier.

So why is the above example behaving differently depending on where ANS() is called?
In reply to Alex Jordan

Re: issue with scaffold.pl or contextForm.pl

by Glenn Rice -
The issue is that the contextForm.pl checker does not preserve state. So when the scaffold executes the answer evaluator to determine if the scaffold should be open it runs the checker once, and changes various internal state variables. Then when the answer evaluator is executed again to actually grade the problem, it again runs the checker. However, now the flags and the internal state variables are not what they were at the beginning, and the checker doesn't work right.

If you add the following after line 79 of the contextForm.pl macro:

my $origContext = Context();
Context($context);
$correct = Formula("$correct");
$student = Formula("$ans->{student_formula}");

and add the following after what is currently line 101 of the macro:

Context($origContext);

then things work as expected.

There may be a better way to do this though.
In reply to Glenn Rice

Re: issue with scaffold.pl or contextForm.pl

by Alex Jordan -
I made a small change to this and things are working, thanks again for the assist. However I hit the same issue with a scaffolded problem that uses contextOrdering.pl (problem file attached). I don't see similarities there with contextForm.pl, and I don't see how to make a similar edit.

I guess I'd like to understand if contextForm and contextOrdering are just badly coded. With the former, it is likely because I made it. With the latter, it is much less likely because Davide made it. If I understood this better, I would be looking for a way to fix this without bumping into it one context file at a time.

In reply to Alex Jordan

Re: issue with scaffold.pl or contextForm.pl

by Glenn Rice -

I am not sure that contextForm.pl is badly coded.  It would be good to figure out what exactly was changed with the contextForm.pl example that is not being properly reverted by scaffold.pl when it executes the answer evaluator.  The approach that I gave really just skirts the issue by saving the current context.

For contextOrdering.pl and the problem you attached, when scaffold.pl executes the answer evaluator the cmp_equal method of contextOrdering.pl is called.  However, the second time it is executed when the answer is actually being graded, that method is not called.  The state of the answer has changed so that its type no longer matches a context::Ordering::Value::Ordering, and it goes to some other value objects cmp_equal method.  I am not sure where yet.

In reply to Glenn Rice

Re: issue with scaffold.pl or contextForm.pl

by Alex Jordan -

Returning to this, I think that it's at the cmp_equal sub defined at line 285 of contextOrdering.pl, that the answer becomes a Value::List object instead of a context::Ordering::Value::Ordering. I'm not sure what to do with that though. I tried using this line:

$self_as_list           = $ans->{correct_value} = Value::List->make($self);

to replace:

$self           = $ans->{correct_value} = Value::List->make($self);

and then a few lines down, using $self_as_list instead of using $self. And that does leave $self as a , context::Ordering::Value::Ordering, but does not fix the issue with the exercise and scaffolding.

In reply to Alex Jordan

Re: issue with scaffold.pl or contextForm.pl

by Glenn Rice -
Having the ANS() call between Section::Begin("") and Section::END() is the same as using PGML. The ANS call is what adds the answer to the PG_ANSWERS_HASH. Note that the "assigned_ans" method of scaffold.pl is called by Section::Begin(""), and the "new_answers" method is called by Section::END().