Your problem is that LimtedPowers::NoBaseE() is not the name of a context, but rather a function that modifies contexts. Calling it causes the current context to be modified, so that
Context("Numeric");
LimitedPowers::NoBaseE();
would modify the Numeric context to not allow e to a power. The reason for this is so that you can modify any context (e.g., Numeric, or Complex, or even Vector) rather than having to have separate contexts for each. Also, the function can take parameters that control the limitations. For example
LimitedPowers::OnlyIntegers(
minPower => -5, maxPower => 5,
message => "integer constants between -5 and 5",
);
allows integer powers only and then only between -5 and 5, and it provides a custom message to issue when the conditions aren't met.
Rather than modifying the current context, you can pass LimitedPowers::NoBaseE() and its brethren a context to operate on. That is how come your second example works. You are passing it a (copy) of the Numeric context, modifying that to not allow powers of e, and then setting it as the current context. That is actually equivalent to my first example above, since setting the context to Numeric already gets you a copy.
Anyway, hope that clear things up.
Davide