Carl K wrote:
> Andrew MacIntyre wrote:
>
> sounds good, but I have 0.0 clue what that actually means.
>
> Can you give me what you did with windows in hopes that I can figure out how to
> do it in Linux? I am guessing it shouldn't be to different. (well, hoping...)

ctypes is a foreign function interface (FFI) extension that became part
of the standard library with Python 2.5 (& is available for 2.3 & 2.4).
It is supported on Linux, *BSD & Solaris (I think) in addition to Windows.

Ghostscript for quite some time has had support for being used as a
library (DLL on Windows). There are only a small number of API functions
exported, and there is information about the net for calling these API
functions from Visual Basic. I wrote a wrapper module using ctypes for
the API based on the C header and the VB information.

To get the best rendering, some understanding of Ghostscript options is
required particularly for image format outputs (eg for anti-aliasing text).

Carl K schrieb:
> Diez B. Roggisch wrote:
>
> server is doing a ton of SQL queries (yes, moving to a 2nd box would be
> nice. might happen mid 2008) so adding HD is an issue. not sure how
> much, but enough to try to avoid it.

Keeping stuff in memory provoking paging isn't?

>
> those split seconds can add up. The server is aleady overloaded, so
> adding more is a big no no.
>> I am willing to take that chance. much better than the 6 hits I know
> would happen using
>
> I have a feeling if I have to create a file, we will go with plan B:
> send the client a pdf and let the user deal with it. Not as nice and
> slick, but won't bog the server.

I have the feeling you just go by your feelings. Which is always a bad
idea regarding performance bottlenecks.

http://en.wikipedia.org/wiki/Optimiz...mputer_science)

So instead of jumping through hoops getting something done the hard way
without knowing how the easy solution affects performance, implement the
feature the easiest way. And SEE if it causes trouble.

Diez

Carl K wrote:
> Rob Wolfe wrote:
>
> This sounds like what I was looking for. some how this got missed when
> I poked around reportlab land.
>
> Thanks much.
>
> Carl K

Beware... AFAIK this is only a backend for reportlab graphics drawings, IOW it
will render drawings and charts from the reportlab.graphics package but will not
render reportlab pdf canvas.

Grant Edwards wrote:
> On 2007-12-25, Diez B. Roggisch <deets> wrote:
>
> Unless the file is really huge (or the server is overloaded),

The server is already overloaded,

> the bytes will probably never even hit a platter. If you're
> using any even remotely modern OS, short-lived tempfiles used
> as you desdcribe are basically just memory-buffers with a
> filesystem API.

Good point. Not that I am willing to risk it (just using the pdf is not such a
bad option) but I am wondering if it would make sense to create a ramdrive for
something like this. if memory is needed, swap would happen, which should be
better than creating files.

Carl K

>>>>> Carl K <carl> (CK) wrote:

>CK> Here is what the code looks like that generates the pdf:

>CK> buffer = StringIO()
>CK> rw = dReportWriter(OutputFile=buffer, ReportFormFile=xmlfile, Cursor=ds)
>CK> rw.write()
>CK> pdf = buffer.getvalue()
>CK> return pdf

You can pipe the pdf through ghostscript and read the png back from
ghostscript's stdout. Like:

gs -q -sDEVICE=png16m -sOutputFile=- -

Use that command in subprocess with the stdin/stdout as pipes, send
your PDF data to the process and read the PNG output back.

However you must be aware that this can deadlock if the output is large
enough. So putting the input or the output in a real file is probably safer
anyway.

>
> Good point. Not that I am willing to risk it (just using the pdf is not
> such a bad option) but I am wondering if it would make sense to create
> a ramdrive for something like this. if memory is needed, swap would
> happen, which should be better than creating files.
>

You mean permanently deprivating your server of it's precious ram than
only temporarily? And so far, swapping has always been the worst thing
to happen for me - whereas mere disk-IO didn't slow _everything_ down.

You should really check your premises on what is affecting performance,
and what not. And get a faster server...


Diez

On Dec 27, 7:33 pm, "Diez B. Roggisch" <de> wrote:
> > Good point.  Not that I am willing to risk it (just using the pdf is not
> > such a bad option)  but I am wondering if it would make sense to create
> > a ramdrive for something like this.  if memory is needed, swap would
> > happen, which should be better than creating files.
>
> You mean permanently deprivating your server of it's precious ram than
> only temporarily? And so far, swapping has always been the worst thing
> to happen for me - whereas mere disk-IO didn't slow _everything_ down.
>
> You should really check your premises on what is affecting performance,
> and what not. And get a faster server...
>
> Diez

What resources this server is lacking currently?
Harddisk space
Harddisk bandwidth (the server just wouldn't stop reading/writing at
peak times)
RAM
Network
CPU

In short, what resource is the most precious and what resources you
still have spares?

Possible option:
* Serve the PDFs directly
--> Harddisk space - low, as it's not converted
--> Harddisk bandwidth - low, depends on how much its requested
--> RAM - low, virtually no RAM usage
--> Network - low, depends on how much its requested
--> CPU - low, virtually no processing is done
--> The simplest to implement, moderate difficulty for users that
don't have PDF reader
--> Users that don't have a PDF reader might experience difficulties,
nowadays most people have Adobe Reader on their computer, and most
people at least know about it. Many computers are also preinstalled
with a pdf reader.

* Preconvert to PNGs, the conversion is done at another computer to
not restrain the server
--> Harddisk space - High, as file is saved as image file
--> Harddisk bandwidth - Moderate, as there are many files and overall
are big
--> RAM - low, virtually no RAM usage
--> Network - Moderate, as the total file served is big
--> CPU - Low, virtually no processing is done
--> Simple

* Serve a precompressed, preconverted PNGs
--> Harddisk space - Moderate-high, should be smaller than just PNG
--> Harddisk bandwidth - Moderate-High, as files can't be served in
chunks
--> RAM - Low, Virtually no RAM usage
--> Network - High, as files can't be served in chunks
--> CPU - Low, virtually no processing is done
--> Moderately-Simple
--> Might be problem if users don't have unzipper (XP, Vista, and most
Linux provides unzipper on default installation, older OSes might not)
--> Files can't be served in chunks, users have to download whole zip
before opening any

* Convert the PDFs to PNG on the fly:
--> Harddisk space - Low, take as much as serving direct PDF
--> Harddisk bandwidth - Moderate to Low, depending on implementation
--> RAM - Moderate, as processing is done, RAM is clearly needed
--> Network - Moderate, as the files served as PNG
--> CPU - High, complex processing is done on the CPU before the file
can be sent to the network
--> Complex to implement

Seeing these options, I think it is much better to serve the PDFs
directly, it's very simple, and very efficient on the server. If
you're afraid that not everyone have PDF readers, direct them to
Adobe's site or serve the installation files on the server. The
installation for the reader is a one-off download, so it should only
choke the server for the first several weeks.

> Seeing these options, I think it is much better to serve the PDFs
> directly, it's very simple, and very efficient on the server. If
> you're afraid that not everyone have PDF readers, direct them to
> Adobe's site or serve the installation files on the server. The
> installation for the reader is a one-off download, so it should only
> choke the server for the first several weeks.

AFAIK the OP wants to render previews for display on the site -
certainly a nice feature.

Diez

Diez B. Roggisch wrote:
>> Seeing these options, I think it is much better to serve the PDFs
>> directly, it's very simple, and very efficient on the server. If
>> you're afraid that not everyone have PDF readers, direct them to
>> Adobe's site or serve the installation files on the server. The
>> installation for the reader is a one-off download, so it should only
>> choke the server for the first several weeks.
>
> AFAIK the OP wants to render previews for display on the site -
> certainly a nice feature.

Exactly.

As far as pre-processing goes: the pdf is generated from data the user just
entered into a web page. it may not even be saved to the DB yet.

the new ImageMagick bindings should do what I need:
http://www.procoders.net/?p=39

Carl K