Forum archive 2000-2006

Jason Farmer - Speed up "typeset2"?

Jason Farmer - Speed up "typeset2"?

by Arnold Pizer -
Number of replies: 0
inactiveTopicSpeed up "typeset2"? topic started 2/13/2003; 11:29:31 PM
last post 7/1/2003; 2:25:38 AM
userJason Farmer - Speed up "typeset2"?  blueArrow
2/13/2003; 11:29:31 PM (reads: 6925, responses: 19)
Hello once again all! It seems as though texhash has removed our "pink" timeout page errors, but typeset2 mode still seems to be taking too long. Under normal load the pages take anywhere from 15 to 30 seconds to generate for a reasonable to small sized problem. We are not getting any "out of the ordinary" error messages in the webwork error_log either. System resources seem to be alright as well. Any ideas on ways to speed this process up?

Thanks once again!!

<| Post or View Comments |>


userArnold K. Pizer - Re: Speed up  blueArrow
2/14/2003; 6:39:25 AM (reads: 7336, responses: 0)
Hi Jason,

I do not know what the problem is but there is a problem (your server can not be that slow). Here is a sample of what our timing_log gives. What does your give?

processProb8.pl (mode: HTML_dpng, action: ) mth162 103-02-14 04:52:21 0 wallclock secs ( 0.27 usr 0.05 sys + 0 .00 cusr 0.00 csys = 0.32 CPU) pid: 9829 DBtie_tries: 7 ck006m

processProb8.pl (mode: HTML_dpng, action: ) mth162 103-02-14 04:52:32 0 wallclock secs ( 0.27 usr 0.05 sys + 0 .00 cusr 0.00 csys = 0.33 CPU) pid: 9835 DBtie_tries: 7 ck006m

processProb8.pl (mode: HTML_dpng, action: Submit Answer) mth162 103-02-14 04:53:09 0 wallclock secs ( 0.24 us r 0.09 sys + 0.00 cusr 0.00 csys = 0.34 CPU) pid: 9836 DBtie_tries: 8 ck006m

processProb8.pl (mode: HTML_tth, action: ) mth163 103-02-14 05:38:08 0 wallclock secs ( 0.28 usr 0.05 sys + 0.01 cusr 0.02 csys = 0.37 CPU) pid: 10477 DBtie_tries: 7 bb005k

processProb8.pl (mode: HTML_tth, action: Submit Answers) mth163 103-02-14 05:44:25 0 wallclock secs ( 0.28 usr 0.06 sys + 0.00 cusr 0.03 csys = 0.38 CPU) pid: 10496 DBtie_tries: 8 bb005k

processProb8.pl (mode: HTML_tth, action: Submit Answers) mth163 103-02-14 05:46:43 1 wallclock secs ( 0.29 usr 0.06 sys + 0.00 cusr 0.03 csys = 0.38 CPU) pid: 10515 DBtie_tries: 8 bb005k

Arnie

<| Post or View Comments |>


userJason Farmer - Re: Speed up  blueArrow
2/14/2003; 9:26:55 AM (reads: 7282, responses: 0)
Here are a couple of our timing logs...

processProb8.pl (mode: HTML_dpng, action: ) Math151-Spring03-NPSigmon 103-02-14 08:23:53 16 wallclock secs ( 1.00 usr 0.08 sys + 11.96 cusr 2.65 csys = 15.69 CPU) pid: 6444 DBtie_tries: 7 cbartys processProb8.pl (mode: HTML_dpng, action: Submit Answer) Math151-Spring03-NPSigmon 103-02-14 08:25:59 1 wallclock secs ( 0.87 usr 0.15 sys + 0.00 cusr 0.00 csys = 1.02 CPU) pid: 6751 DBtie_tries: 7 cbartys processProb8.pl (mode: HTML_dpng, action: Submit Answer) Math151-Spring03-NPSigmon 103-02-14 08:27:30 1 wallclock secs ( 0.95 usr 0.11 sys + 0.00 cusr 0.00 csys = 1.06 CPU) pid: 6752 DBtie_tries: 7 cbartys processProb8.pl (mode: HTML_tth, action: ) Math121-Spring03-CMett 103-02-14 09:20:52 1 wallclock secs ( 0.66 usr 0.07 sys + 0.00 cusr 0.00 csys = 0.73 CPU) pid: 6771 DBtie_tries: 2 cmett login.pl Math121-Spring03-CMett 103-02-14 09:20:56 0 wallclock secs ( 0.21 usr 0.02 sys + 0.00 cusr 0.00 csys = 0.23 CPU) pid: 6772 DBtie_tries: 3 cmett 137.45.240.17 Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; Q312461; (R1 1.1)) processProb8.pl (mode: HTML_tth, action: ) Math121-Spring03-CMett 103-02-14 09:21:00 2 wallclock secs ( 1.03 usr 0.20 sys + 0.19 cusr 0.16 csys = 1.58 CPU) pid: 6773 DBtie_tries: 7 cmett

<| Post or View Comments |>


userJason Farmer - Re: Speed up  blueArrow
2/14/2003; 9:30:21 AM (reads: 7349, responses: 0)
Here are a couple of our timing logs...

processProb8.pl (mode: HTML_dpng, action: ) Math151-Spring03-NPSigmon 103-02-14 08:23:53 16 wallclock secs ( 1.00 usr 0.08 sys + 11.96 cusr 2.65 csys = 15.69 CPU) pid: 6444 DBtie_tries: 7 cbartys processProb8.pl (mode: HTML_dpng, action: Submit Answer) Math151-Spring03-NPSigmon 103-02-14 08:25:59 1 wallclock secs ( 0.87 usr 0.15 sys + 0.00 cusr 0.00 csys = 1.02 CPU) pid: 6751 DBtie_tries: 7 cbartys processProb8.pl (mode: HTML_dpng, action: Submit Answer) Math151-Spring03-NPSigmon 103-02-14 08:27:30 1 wallclock secs ( 0.95 usr 0.11 sys + 0.00 cusr 0.00 csys = 1.06 CPU) pid: 6752 DBtie_tries: 7 cbartys processProb8.pl (mode: HTML_tth, action: ) Math121-Spring03-CMett 103-02-14 09:20:52 1 wallclock secs ( 0.66 usr 0.07 sys + 0.00 cusr 0.00 csys = 0.73 CPU) pid: 6771 DBtie_tries: 2 cmett login.pl Math121-Spring03-CMett 103-02-14 09:20:56 0 wallclock secs ( 0.21 usr 0.02 sys + 0.00 cusr 0.00 csys = 0.23 CPU) pid: 6772 DBtie_tries: 3 cmett 137.45.240.17 Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; Q312461; (R1 1.1)) processProb8.pl (mode: HTML_tth, action: ) Math121-Spring03-CMett 103-02-14 09:21:00 2 wallclock secs ( 1.03 usr 0.20 sys + 0.19 cusr 0.16 csys = 1.58 CPU) pid: 6773 DBtie_tries: 7 cmett

<| Post or View Comments |>


userJohn Jones - Re: Speed up  blueArrow
2/14/2003; 9:40:51 AM (reads: 7335, responses: 0)
Hi,

The first thing I would try is:

  • don't let the system delete the .tex files. I think setting imageDebugMode in Global.pm will take care of that
  • view a problem, and then go look for the .tex file in the tmp directory of that course
  • run latex on the file. Then run dvipng on the dvi file. See if either operation takes a long time, or fails. In the end, you should have png files.
  • delete the dvi and png files and repeat.

The reason for doing repeating is that dvipng may need to build fonts the first time it is run. If you see metafont being invoked, then ignore that run and create the files again. (If metafont keeps getting called, then you have found the problem.)

When looking at the output from the latex run, there will be things which may look like errors coming from preview.sty, but which are really diagnostics it produces under normal operation.

John

<| Post or View Comments |>


userJason Farmer - Re: Speed up  blueArrow
2/14/2003; 9:47:12 AM (reads: 7348, responses: 0)
I will give that a try. Thank you all.

<| Post or View Comments |>


userJason Farmer - Re: Speed up  blueArrow
2/23/2003; 10:42:04 PM (reads: 7325, responses: 0)
The dvipng seems to take a long time when invoked from the command prompt as well, and Metafont is being called each time dvipng is run. What is Metafont doing, and how might I go about fixing this?

Thanks once again.

<| Post or View Comments |>


userJohn Jones - Re: Speed up  blueArrow
2/24/2003; 6:56:09 PM (reads: 7289, responses: 0)
Hi,

TeX uses bitmapped fonts. In general, fonts of this kind don't scale very well (jagged edges), so Knuth also invented metafont. It is essentially a language for outline type fonts (which pre-dates postscript fonts and truetype fonts). The metafont program can take these outline versions of a font and build a full set of bitmapped files in a desired size, and tuned to a particular output device.

Many TeX systems are set up to build new bitmap font files on demand. Once built, the system uses them. Typically, you use the same fonts over and over again, so new building of the bitmap files becomes less and less frequent.

The fact that metafont keeps on getting called is a sign that either the bitmap font creation is failing for some reason (permissions of the directory where the new files go?), or the newly built files are not being found and reused.

Here are some questions which come to mind:

What is the operating system (and distribution, if it is linux)?

Was something special done as far as the installation of TeX (e.g., installing a "better" version than whatever came on the system)?

Does font creation work properly with LaTeX/dvips? To test this, you could try a small document of documentclass book with the 11pt option, and then use various font changing commands (set the size huge and/or tiny, and include boldface and/or italic bits of text). Actually, if you make just one letter in a funny size/face, that is best since metafont will only try to build one font.

Run latex, then dvips. Metafont should be triggered. Look for errors or a file called missfont.log about missing fonts. Then run dvips again on the same document. Metafont should not run, but if it does, you can try to see what font file it is looking for, and if it exists elsewhere on your disk.

John

<| Post or View Comments |>


userJason Farmer - Re: Speed up  blueArrow
2/25/2003; 7:37:44 PM (reads: 7299, responses: 0)
First off I would like to just put in a little input and thank you so much for your help John.

We are not running Linux, but Solaris 8. Most all of the software was installed using default options. However, some modifications have been done to latex2html to get it to work properly (ie not display the "black boxes" around images). When you mentioned the missfont.log, I do remember that log file being generated, and actually containing quite a few fonts. Here is an excerpt from one:

mktexpk --mfmode ljfour --bdpi 600 --mag 1+0/600 --dpi 600 cmex10 mktexpk --mfmode ljfour --bdpi 600 --mag 0+444/600 --dpi 444 cmmi10 mktexpk --mfmode ljfour --bdpi 600 --mag 1+0/600 --dpi 600 cmex10 mktexpk --mfmode ljfour --bdpi 600 --mag 0+444/600 --dpi 444 cmr10 mktexpk --mfmode ljfour --bdpi 600 --mag 1+0/600 --dpi 600 cmex10 mktexpk --mfmode ljfour --bdpi 600 --mag 0+444/600 --dpi 444 cmr10 mktexpk --mfmode ljfour --bdpi 600 --mag 1+0/600 --dpi 600 cmex10 mktexpk --mfmode ljfour --bdpi 600 --mag 0+444/600 --dpi 444 cmr10 mktexpk --mfmode ljfour --bdpi 600 --mag 1+0/600 --dpi 600 cmex10 mktexpk --mfmode ljfour --bdpi 600 --mag 0+444/600 --dpi 444 cmr10 mktexpk --mfmode ljfour --bdpi 600 --mag 1+0/600 --dpi 600 cmex10 mktexpk --mfmode ljfour --bdpi 600 --mag 0+444/600 --dpi 444 cmr10 mktexpk --mfmode ljfour --bdpi 600 --mag 1+0/600 --dpi 600 cmex10 mktexpk --mfmode ljfour --bdpi 600 --mag 0+444/600 --dpi 444 cmr10 mktexpk --mfmode ljfour --bdpi 600 --mag 1+0/600 --dpi 600 cmex10 mktexpk --mfmode ljfour --bdpi 600 --mag 0+444/600 --dpi 444 cmmi10 mktexpk --mfmode ljfour --bdpi 600 --mag 1+0/600 --dpi 600 cmex10 mktexpk --mfmode ljfour --bdpi 600 --mag 0+444/600 --dpi 444 cmr10 mktexpk --mfmode ljfour --bdpi 600 --mag 1+0/600 --dpi 600 cmex10 mktexpk --mfmode ljfour --bdpi 600 --mag 0+444/600 --dpi 444 cmr10 mktexpk --mfmode ljfour --bdpi 600 --mag 1+0/600 --dpi 600 cmex10 mktexpk --mfmode ljfour --bdpi 600 --mag 0+444/600 --dpi 444 cmr10 mktexpk --mfmode ljfour --bdpi 600 --mag 1+0/600 --dpi 600 cmex10

And here is another mktexpk --mfmode ljfour --bdpi 600 --mag 1+0/600 --dpi 600 cmex9 mktexpk --mfmode ljfour --bdpi 600 --mag 1+0/600 --dpi 600 cmex9 mktexpk --mfmode ljfour --bdpi 600 --mag 1+0/600 --dpi 600 cmex9

So it seems like we are missing quite a few fonts. Is the last part of this statement (for instance the "cmex9" the font?

When I do a command line call of dvipng on a dvi file, this is an excerpt from some of the output that I get as well:

kpathsea: Running mktexpk --mfmode cx --bdpi 300 --mag 1+29/300 --dpi 329 ptmb7t mktexpk: Running mf mode:=cx; mag:=1+29/300; nonstopmode; input ptmb7t This is METAFONT, Version 2.7182 (Web2C 7.3.1)

kpathsea: Running mktexmf ptmb7t ! I can't find file `ptmb7t'. <*> ...x; mag:=1+29/300; nonstopmode; input ptmb7t Please type another input file name ! Emergency stop. <*> ...x; mag:=1+29/300; nonstopmode; input ptmb7t

That might help some. Note that this is from a different file from the missfont.log's above.

I really hope that we can find a solution to what is causing our problem. Thank you once again for your help.

Jason

<| Post or View Comments |>


userJohn Jones - Re: Speed up  blueArrow
2/25/2003; 10:26:33 PM (reads: 7307, responses: 0)
Hi,

Yes, cmex9 is a font name. It is size 9 of computer modern ex (I forget what ex is for).

As root, try

find / -name "cmex9*"

This will search your entire disk for files whose names start with cmex9, so it may take a minute or two. You are particularly interested in the file cmex9.mf, which is the metafont source file (which is aparantly not found in one of the other examples).

So, if it doesn't find cmex9.mf, you don't have the necessary metafont files to build fonts. If it exists, then the problem is just that your system isn't finding it. My copy is in .../texmf/fonts/source/public/cm/cmex9.mf, but these things vary.

We are pushing the limits of what I know about TeX's operation, but I think .../texmf/web2c/texmf.cnf is the file which sets initial paths for things like file searching. I think the relevant line in my file is

 

 MFINPUTS = .;$TEXMF/metafont//;{$TEXMF/fonts,$VARTEXFONTS}/source//

which tels it to look in all subdirectories (final two slashes) of .../texmf/fonts/source, amoung other places.

John

I don't have a solaris machine. Maybe someone who does can tell us where things should be.

<| Post or View Comments |>


userJason Farmer - Re: Speed up  blueArrow
2/25/2003; 11:40:23 PM (reads: 7415, responses: 0)
Well a good reason why TeX is not finding this font is that we do not have it! Wow if all of our problems boil down to a missing font package, then I am going to be very thankful!! I searched for that particular font, and it was not found on the drive anywhere. I also searched on the internet and could not find where to download this font. Any ideas in which package this font might be?

Jason

<| Post or View Comments |>


userJohn Jones - Re: Speed up  blueArrow
2/26/2003; 12:14:40 PM (reads: 7381, responses: 0)
It is on CTAN (http://www.ctan.org) under fonts/cm/mf. cm will cover most of your missing font needs, but you may need to get other collections too. For example, your output above showed ptmb7t was missing, although I don't think that is from WeBWorK since it is an adobe times font.

John

<| Post or View Comments |>


userJason Farmer - Re: Speed up  blueArrow
2/26/2003; 1:24:45 PM (reads: 7323, responses: 0)
I am downloading some of the fonts from CTAN now, and I am putting them in /texmf/metafont/ which seems to be an appropiate directory. Any directory that was listed in the texmf.cnf would be fine correct? And once I download these fonts, will a simple, `texhash` update the texmf fonts?

<| Post or View Comments |>


userJason Farmer - Re: Speed up  blueArrow
2/26/2003; 1:46:01 PM (reads: 7325, responses: 0)
Once again webwork discussion board has come to the rescue!

It appears as though what I did in the above post is working! Of course I do not think Webwork is under a heavy load, but I am not fully convinced until I see what performance is like under a very heavy load. I am going to keep a close eye on this, but I do feel confident that this may have solved our problem! I am very thankful it was something as simple as missing fonts!

Thank you very much for your help John!

Take care,

Jason

P.S. I may be back! :)

<| Post or View Comments |>


userJason Farmer - Re: Speed up  blueArrow
4/10/2003; 12:14:42 AM (reads: 7221, responses: 0)
It appears as though we are getting the "Pink Pages" once again. After reviewing the error_log.log under webwork/system/logs, I notice this output for every typeset2 conversion:

kpathsea: Running mktexpk --mfmode toshiba --bdpi 180 --mag 4+0/180 --dpi 720 cmr12 mktexpk: /home/webwork2/local/teTeX/share/texmf/fonts/pk/toshiba/public/cm/cmr12.720pk already exists. kpathsea: Running mktexpk --mfmode toshiba --bdpi 180 --mag 4+0/180 --dpi 720 cmmi12 mktexpk: /home/webwork2/local/teTeX/share/texmf/fonts/pk/toshiba/public/cm/cmmi12.720pk already exists.

The only differences is that there are a variety of the .720pk files specified. I believe I have solved the problem of missing fonts, but it still appears as though kpathsea and mktexpk are running constantly.

Any other ideas on how I might find out some more information?

Thanks again,

Jason

<| Post or View Comments |>


userJan-Åke Larsson - Re: Speed up  blueArrow
6/17/2003; 5:04:58 AM (reads: 7162, responses: 0)
It seems as if kpathsea doesn't find the (already existing) cmr12.720pk. Try

ls /home/webwork2/local/teTeX/share/texmf/fonts/pk/toshiba/public/cm/cmr12.720pk

and

kpsewhich -m cmr12.720pk

If the first finds the file, and the second does not, something is not right in your setup. Some path is missing in kpathsea. If kpsewhich reports the right file, something else is wrong, but try this first.

Jan-Åke

<| Post or View Comments |>


userJason Farmer - Re: Speed up  blueArrow
6/29/2003; 8:46:50 PM (reads: 7121, responses: 0)
I tried running the kpsewhich -m cmr12.720pk command that you advised and it came back with the following : kpsewhich: option `-m' is ambiguous

Jason

<| Post or View Comments |>


userJan-Åke Larsson - Re: Speed up  blueArrow
6/30/2003; 6:44:49 AM (reads: 7061, responses: 0)
Sorry, that should have been

kpsewhich -mode=toshiba cmr12.720pk

You could try just

kpsewhich cmr12.720pk

Jan-Åke

<| Post or View Comments |>


userJason Farmer - Re: Speed up  blueArrow
6/30/2003; 6:19:28 PM (reads: 7084, responses: 0)
Running both of those commands finds the file: /home/webwork2/local/teTeX/share/texmf/fonts/pk/toshiba/public/cm/cmr12.720pk

It took a little while to run also, so I decided to time it and it produced in 3.21 seconds. So it wasn't abnormally slow, considering our pages take in upwards of 30 seconds to load.

<| Post or View Comments |>


userJan-Åke Larsson - Re: Speed up  blueArrow
7/1/2003; 2:25:38 AM (reads: 7021, responses: 0)
There are then two problems: dvipng seems not to find the rendered font, and kpathsea takes a long time.

The dvipng problem: I suggest you send me the output of using "dvipng -d", possibly "-pp 1" or whatever page the problem is on, on the offending DVI, _via_email_, don't post it on this page. We'll try and debug this off-list.

The long path-search-time, well, you have a fast machine, right? And the /home/webwork2 dir is not on NFS, right? (If it is, the PK fonts are better off in /tmp, which is on the local hard drive=more speed.) And don't put more in FONTPATH than is necessary. The following is an excerpt from the info manual of kpathsea. -----------------------------------------------


Slow path searching ...................

If your program takes an excessively long time to find fonts or other input files, but does eventually succeed, here are some possible culprits:

* Most likely, you just have a lot of directories to search, and that takes a noticeable time. The solution is to create and maintain a separate `ls-R' file that lists all the files in your main TeX hierarchy. *Note Filename database::. Kpathsea always uses `ls-R' if it's present; there's no need to recompile or reconfigure any of the programs.

* Your recursively-searched directories (e.g., `/usr/local/share/texmf/fonts//'), contain a mixture of files and directories. This prevents Kpathsea from using a useful optimization (*note Subdirectory expansion::.).

It is best to have only directories (and perhaps a `README') in the upper levels of the directory structure, and it's very important to have *only* files, and no subdirectories, in the leaf directories where the dozens of TFM, PK, or whatever files reside.

In any case, you may find the debugging options helpful in determining precisely when the disk or network is being pounded. *Note Debugging::.

<| Post or View Comments |>