I'm using AJAX on a Nano54415 with NNDK rev 2.6.5. I'm finding that when I transfer a large file (multipart post), the AJAX times out (which makes sense because the web server is busy for 30s with the file upload). I was looking into modifying the multipart handler to pass the socket to a worker thread, but realized a multi-thread server would be more generic and perhaps better.
Right now I'm assuming I'm going to implement it myself, and I'm looking for tips/tricks/pointers.
Anybody done anything similar?
Thanks!
multithreading web server?
Re: multithreading web server?
Yes. I'll post my layout for an overhaul later tonight or this weekend. This has been on my todo list for quite some time...
Dan Ciliske
Project Engineer
Netburner, Inc
Project Engineer
Netburner, Inc
Re: multithreading web server?
So, let's begin. First, a multithreaded/multitasked webserver implemented naively on the Netburner OS will never work the way you intend due to the nature of scheduling. The OS relies on preemptive scheduling; the highest priority task that wants to run, runs. Further complicating things is that what you really want is not sharing clock cycles across multiple request handlers, but rather network bandwidth. How in reality would we implement bandwidth allocations? Here's where things start to get complicated.
For the sake of argument, let's assume the simplest case: we're connected only to a LAN where our 100Mb Full duplex link is the smallest pipe. When we get an incoming connection (a successful return from accept), we hand of the socket number to a waiting worker task. This task then handles the response and starts sending the response. Let's say architecturally that this is done by sticking the fd num into a OS_Q and having the handlers pend on the queue. In the naive case, the first handler runs until its TCP socket is out of space whether due to actually being windowed off or latency delaying the acks. At that point this first handler pends and finally our second handler gets to run. But here's the thing: there's no guarantee that this second handler ever does run before the first handler finishes. So we need some way to actually limit how much the first handler can send before we let the second handler run.
To do this we need to do one of two things: create a new type of fd that can implement bandwidth caps or add bandwidth caps to the core TCP sockets. The basic implementation is pretty straight forward. Add a variable that tracks how much space we have left to send in our allocation and when it hits 0 we don't get to write any more, until receiving more credits. This is effectively how the TCP protocol operates, where the new credits are given by the remote end. In our bandwidth sharing case, we would probably like this to be done on a pool basis. What this likely becomes is a flag being added in the core TCP that determines whether we allow writing out the TCP frames on our end and adding/modifying a flush function to send the pending data. In this case the task pending is already built into the socket structure. We can then implement whatever bandwidth scheme we wish inside the main HTTP task when it determines which sockets to flush.
Uhh... I'm currently working on an issue with TLS, but this will be my next item of development. What's your timetable? When would you need/want this by?
-Dan
For the sake of argument, let's assume the simplest case: we're connected only to a LAN where our 100Mb Full duplex link is the smallest pipe. When we get an incoming connection (a successful return from accept), we hand of the socket number to a waiting worker task. This task then handles the response and starts sending the response. Let's say architecturally that this is done by sticking the fd num into a OS_Q and having the handlers pend on the queue. In the naive case, the first handler runs until its TCP socket is out of space whether due to actually being windowed off or latency delaying the acks. At that point this first handler pends and finally our second handler gets to run. But here's the thing: there's no guarantee that this second handler ever does run before the first handler finishes. So we need some way to actually limit how much the first handler can send before we let the second handler run.
To do this we need to do one of two things: create a new type of fd that can implement bandwidth caps or add bandwidth caps to the core TCP sockets. The basic implementation is pretty straight forward. Add a variable that tracks how much space we have left to send in our allocation and when it hits 0 we don't get to write any more, until receiving more credits. This is effectively how the TCP protocol operates, where the new credits are given by the remote end. In our bandwidth sharing case, we would probably like this to be done on a pool basis. What this likely becomes is a flag being added in the core TCP that determines whether we allow writing out the TCP frames on our end and adding/modifying a flush function to send the pending data. In this case the task pending is already built into the socket structure. We can then implement whatever bandwidth scheme we wish inside the main HTTP task when it determines which sockets to flush.
Uhh... I'm currently working on an issue with TLS, but this will be my next item of development. What's your timetable? When would you need/want this by?
-Dan
Dan Ciliske
Project Engineer
Netburner, Inc
Project Engineer
Netburner, Inc
Re: multithreading web server?
Hmm - seems there's a login timeout or something, so I can't take 3 hours to edit the post like I usually do (without writing in notepad and cut/paste into IE).
Short recap of what I just tried to post (which means it's more concise, less rambling, and not wasted):
Your use case is a heavily-loaded web server (showing lag serving pages, perhaps) where you have to throttle the served pages in order for worker tasks to make sense. Because we can't really time-slice the CPU among equally-prioritized tasks, that requires bandwidth throttling. This is more than I need.
My use case is http file transfer with simultaneous AJAX requests. All I really need is to pass the file transfer to a work thread. I saw handing the socket to a work task from within the multipart handler as a possible problem, and what I really wanted to ask was how far back up the call stack I should try to go in order to pass things to worker tasks. You answered my question, meaning that now I realize I don't _really_ want to attempt a proper multithreading web server. I had not thought the problem all the way through.
I think that if I pass the post/multipart handler calls to a lower-priority work task, I should be able to do what I need without attempting network bandwidth allocation. I want to serialize the handler calls in order to avoid some multithreading issues, so there's a limit to how many work tasks I actually can use. I could defer the get handler calls to a work task as well, but I'm not decided on that point - looks like I could pass the socket to a work thread from inside the gethandler callback, as long as I fix the return code.
The upside is that I can keep using the existing handler code. The downside is that I will still see the same delay if two clients post at the same time. That's OK for my use case, not necessarily good enough for someone else's.
Thanks!
Short recap of what I just tried to post (which means it's more concise, less rambling, and not wasted):
Your use case is a heavily-loaded web server (showing lag serving pages, perhaps) where you have to throttle the served pages in order for worker tasks to make sense. Because we can't really time-slice the CPU among equally-prioritized tasks, that requires bandwidth throttling. This is more than I need.
My use case is http file transfer with simultaneous AJAX requests. All I really need is to pass the file transfer to a work thread. I saw handing the socket to a work task from within the multipart handler as a possible problem, and what I really wanted to ask was how far back up the call stack I should try to go in order to pass things to worker tasks. You answered my question, meaning that now I realize I don't _really_ want to attempt a proper multithreading web server. I had not thought the problem all the way through.
I think that if I pass the post/multipart handler calls to a lower-priority work task, I should be able to do what I need without attempting network bandwidth allocation. I want to serialize the handler calls in order to avoid some multithreading issues, so there's a limit to how many work tasks I actually can use. I could defer the get handler calls to a work task as well, but I'm not decided on that point - looks like I could pass the socket to a work thread from inside the gethandler callback, as long as I fix the return code.
The upside is that I can keep using the existing handler code. The downside is that I will still see the same delay if two clients post at the same time. That's OK for my use case, not necessarily good enough for someone else's.
Thanks!
Re: multithreading web server?
Oops - I don't need to fix the return code for GetHandler, I just have to return the correct return code from my callback. Sorry.