Welcome, Guest. Please Login or Register
YaBB - Yet another Bulletin Board
09.02.2010 at 19:20:35
News: Server upgrade went fine, you are now at the new system


Pages: 1 2 
Problem with multpile (fast) client-requests (Eth) (Read 3840 times)
Douwe Homans
YaBB Newbies
*
Offline

Arduino rocks

Posts: 6

Problem with multpile (fast) client-requests (Eth)
24.02.2009 at 17:53:26
 
Hello!

I'm new to Arduino, and want to do the following.
I want simple board with a button, when i press the button i want it to do a http-request, and display contents on the mac using serial connection.

This shouldn't be too hard..

I thought I'd best approach this problem one sub-problem at the time.
My first step was to create the http-request and serial display.

I found the code below, and understand what it does..
I inserted the delay(500); pieces myself.

Problem with code below is that sometimes it can't connect to the server, I don't understand why cause it really is up. When I increase the amount of delay it seems to go better. But I want to be able to push the button (in the final) project multpile times per second....

This should be possible, shouldn't is? Could the problem be serverside? If anyone can help, it would help me a lot.

Thanks!

Code:
#include <Ethernet.h>

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
byte ip[] = { 10,0,0,7 };
byte server[] = {10,0,0,1};


Client client(server, 80);

enum CLIENT_STATE
{
	STEP1, STEP2, STEP3, STEP4
};

static CLIENT_STATE client_state;

void setup()
{
  Ethernet.begin(mac, ip);
  Serial.begin(9600);

  delay(1000);
}

void loop()
{

  if (client_state==STEP1)
  {
    Serial.println("connecting...");

  if (client.connect()) {
    Serial.println("connected");

    client.println("GET /index.html HTTP/1.0");
    client.println("Host: localhost");
    client.println("User-Agent: AVR ethernet");
    client.println("Accept: text/html");

    client.println();


    client_state=STEP2;
    delay(500);
  } else {
    Serial.println("connection failed");
    delay(500);
    client_state=STEP1;
  }
  }

    if (client_state==STEP2)
  {
  if (client.available()) {
    char c = client.read();
    Serial.print(c);
    client_state=STEP2;
  }
  else if (!client.connected())
  {    Serial.println();
    Serial.println("disconnecting.");
    client.stop();
    delay(500);
	client_state=STEP1;
    }

  }
}
 

Back to top
 
 
View Profile   IP Logged
jerrya.net
Junior Member
**
Offline

Arduino rocks

Posts: 83
St. Louis MO
Re: Problem with multpile (fast) client-requests (Eth)
Reply #1 - 24.02.2009 at 20:59:26
 
I've been having a similar problem, but not related to the speed of requests.

I have mine set up to pull a web page every 60 sec, and I have an LED indicate when there was a connection failure. Maybe I should log the failures so I know for sure how often they happen, but it seems to have a fair number of them throughout the day.

The server is of course up and responding, but I'm not sure what to look at in my Arduino to see what the problem is.
Back to top
 
 
View Profile   IP Logged
Douwe Homans
YaBB Newbies
*
Offline

Arduino rocks

Posts: 6

Re: Problem with multpile (fast) client-requests (Eth)
Reply #2 - 25.02.2009 at 16:08:05
 
I've rewritten the code. Now it is only the essence...
I do a request every second with the doRequest function... It works a couple of times, but then it suddenly can't connect anymore (i keep getting "Can't get a connection!").

After a while it suddenly can connect again, and i get results from te server again. Most of the time it works once or twice, and then the procees repeats... (can't get a connection, and after a while CAN get a connection.)

Does anyone has an idea where the problem could be?
Code:
#include <Ethernet.h>

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
byte ip[] = { 10, 0, 0, 12 };
byte server[] = {10,0,0,1 }; // local ip

Client client(server, 80);

void setup()
{
  Ethernet.begin(mac, ip);
  Serial.begin(9600);
}

void loop()
{
  doRequest();
  delay(1000);
}

void doRequest(){

  Serial.println("doRequest Function");
  while(!client.connect()) {
    Serial.println("Can't get a connection!");
  }
  Serial.println("(Finally) got a connection, so send get-request.....");
   client.println("GET /index.html HTTP/1.0");
   client.println();
  Serial.println("Reading incoming data");
  incoming();
  client.stop();
  Serial.println('socket stop');
}

void incoming(){
  while (client.available()) {
    char c = client.read();
    Serial.print(c);
  }

  if (!client.connected()) {
    Serial.println("Disconnected.");
  }
}
 



Back to top
 
 
View Profile   IP Logged
Mark Bramwell
Junior Member
**
Offline

Arduino rocks

Posts: 69
London, Ontario, Canada
Re: Problem with multpile (fast) client-requests (Eth)
Reply #3 - 25.02.2009 at 20:32:18
 
Any chance the connection is not fully stopped between tries?

Instead of relying on a delay(), try checking the status of the connection.

something like so:

while ( client.connected )
{
client.stop();
Serial.println('stopping the connection... ');
}

Serial.println('socket stop');


I have not played with the ethernet lib yet. Does the status change to disconnected right away or does it takes a few moments depending on the response of the remote machine?
Does the Stopping....  message always print exactly once?

In vb6, I have found that it takes time for the connection to fully stop after I issue the disconnect request. I have not yet played with my ethernet shield. Ideally you would say  while ( !client.disconnected ) but I have not verified the types of 'states' that the library provides.

example:  'not connected' is not the same as 'disconnected' because we could be in transition of states.

Back to top
 
 
View Profile | WWW   IP Logged
NJPaul
YaBB Newbies
*
Offline

Arduino rocks

Posts: 2

Re: Problem with multpile (fast) client-requests (Eth)
Reply #4 - 28.03.2009 at 22:36:44
 
Mark,

Looks like you've found the solution. I was having the same exact issue. Putting in a delay is not reliable. Instead, looping until the connection ends works like a charm. Thanks!

Paul

Back to top
 
 
View Profile   IP Logged
macegr
God Member
*****
Offline

Arduino Ninja

Posts: 689

Re: Problem with multpile (fast) client-requests (Eth)
Reply #5 - 28.03.2009 at 22:51:54
 
This post deals with the same problem in a slightly different way: http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1235991468/14#14

Looping until !connected() can still cause problems, because of the way the Ethernet library determines connection status.
Back to top
 
 
View Profile | WWW   IP Logged
XNDR
YaBB Newbies
*
Offline

Arduino rocks

Posts: 23

Re: Problem with multpile (fast) client-requests (Eth)
Reply #6 - 29.03.2009 at 04:04:03
 
I'm having similar problems... did you ever figure out how to fix this?

Here's some more information about this problem:
http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1238295170
Back to top
 
 
View Profile   IP Logged
NJPaul
YaBB Newbies
*
Offline

Arduino rocks

Posts: 2

Re: Problem with multpile (fast) client-requests (Eth)
Reply #7 - 29.03.2009 at 15:57:26
 
In addition to the loop to check whether the connection has been closed, I noticed that a few well placed delay() statements are necessary. Otherwise, after 4 or 5 connections the ethernet board 'locks up'. More specifically, the tx LED on the ethernet board goes nuts but othewise all communications cease. Here is my code after I put in the delay statements. Works fine now. Disclaimer: I am very new to Arduino.

#include <Ethernet.h>

byte mac[] = {
 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
byte ip[] = {
 x, x, x, x };
byte server[] = {
 y, y, y, y };
Client client(server, 86);


int sen02=0;

void setup()
{
 Ethernet.begin(mac, ip);
 pinMode(2, INPUT);
 delay(1000);
}

void loop()
{

if(digitalRead(2)!=sen02){
sen02=digitalRead(2);
 client.connect();
 delay(1000);
 if(client.connected(){
   client.println("GET /HA/Event.php?Sen=2&Sta=1&Des=Text1 HTTP/1.0") ;
   delay(500);
   client.println();
   delay(500);
   while(client.connected()) {
     client.flush();
     client.stop();
     delay(1000);
     }
   }
 
}

}
Back to top
 
 
View Profile   IP Logged
etracer
Full Member
***
Offline

Arduino rocks

Posts: 233
Indiana
Gender: male
Re: Problem with multpile (fast) client-requests (Eth)
Reply #8 - 31.03.2009 at 03:23:37
 
Here's a fix (at least a work-around) to the multiple connect problem. The problem seems to be in how the TCP stack on the Wiznet chip is handling disconnects. It looks like it's failing to ACK the final FIN when the connection is closing. This leaves the socket in the SOCK_FIN_WAIT state from the client (Wiznet chip), and in the TIME_WAIT state on the server. Eventually these states timeout after about 30 seconds and the socket moves to a SOCK_CLOSED state.

So as you initiate each connection and subsequently close it (or it gets closed by the webserver), the one of the four available sockets for connections gets "stuck" waiting for the TCP connection to close (because of the FIN/ACK issue). After four connections in less than about 30 seconds, all of the sockets are in use (stuck waiting) and no more connections can be established. As each stock socket times out, then another connection can be made.

So the work-around is to change the Client connect() function to consider a socket in the SOCK_FIN_WAIT state available. This gets around the 30 second delay while the state times out. This solution is not ideal as it's not very nice to the server you connected to since it will end up with a bunch of TIME_WAIT connections that will eventually timeout. This could be bad if you make a lot of connections very quickly. It's what's happening now anyway, but you can only generate 4 at a time so it's not so bad on the server. The real fix is probably needed in the Wiznet TCP/IP stack on the chip - maybe there's an updated firmware??

Here's the revised connect() function in Client.cpp. Note that there are a couple of other enhancements to:
1. More intelligently choose the available socket instead of always picking the last one.
2. Bugfix for the _srcport overflowing and causing the connection to come from a privileged port.

Code:
uint8_t Client::connect() {

  uint8_t socketStatus;
  uint8_t i=0;

  _sock = 255;

  while ((i < MAX_SOCK_NUM) && (_sock == 255)) {
    socketStatus = getSn_SR(i);
    if ((socketStatus == SOCK_CLOSED) || (socketStatus == SOCK_FIN_WAIT)) {
      _sock = i;
    } else {
      i++;
    }
  }
  
  if (_sock == 255)
    return 0;
    
  _srcport++;

  if (_srcport > 64511)
    _srcport = 0;
    
  socket(_sock, Sn_MR_TCP, 1024 + _srcport, 0);
  
  if (!::connect(_sock, _ip, _port))
    return 0;
    
  while (status() != SOCK_ESTABLISHED) {
    if (status() == SOCK_CLOSED)
      return 0;
  }
  
  return 1;
} 



Also, if we're going to consider the SOCK_FIN_WAIT state to be disconnected and eligible for new connections, the we probably need to update the connected() function as well:

Code:
uint8_t Client::connected() {
  uint8_t s = status();
  return !(s == SOCK_LISTEN || s == SOCK_CLOSED || (s == SOCK_CLOSE_WAIT && !available()) || (s == SOCK_FIN_WAIT));
} 



With these changes I was able to make unlimited client connections to a webserver.
Back to top
 
 
View Profile   IP Logged
macegr
God Member
*****
Offline

Arduino Ninja

Posts: 689

Re: Problem with multpile (fast) client-requests (Eth)
Reply #9 - 31.03.2009 at 04:02:43
 
That's exactly what I was seeing, my fix was just to actually wait until the socket was closed. It still let me get a few requests per second.

Great work! The Wiznet chip is amazing, it's bad to see so much of its power go to waste. Wiznet supplies code for DHCP, which might be interesting to get running sometime.
Back to top
 
 
View Profile | WWW   IP Logged
XNDR
YaBB Newbies
*
Offline

Arduino rocks

Posts: 23

Re: Problem with multpile (fast) client-requests (Eth)
Reply #10 - 01.04.2009 at 12:26:41
 
It looks like you guys fixed the problem!

I'm using code based on the example posted by NJPaul and the client.cpp modifications suggested by etracer, great work!

Oh and this patch is meant for the standard Ethernet library, it doesn't work with Ethernet2 yet.

Will this patch find it's way to the official distribution?

Back to top
 
 
View Profile   IP Logged
sarmisak
YaBB Newbies
*
Offline

Arduino rocks

Posts: 5

Re: Problem with multpile (fast) client-requests (Eth)
Reply #11 - 01.04.2009 at 12:52:29
 
I have applied the patches by etracer and used the example code by NJPaul, nothing has changed. I still get the same SYN, RST in wireshark session logs.
Back to top
 
 
View Profile   IP Logged
etracer
Full Member
***
Offline

Arduino rocks

Posts: 233
Indiana
Gender: male
Re: Problem with multpile (fast) client-requests (Eth)
Reply #12 - 01.04.2009 at 13:21:03
 
Did you remember to delete the Client.o file in the library directory? If not, the the library didn't recompile.
Back to top
 
 
View Profile   IP Logged
sarmisak
YaBB Newbies
*
Offline

Arduino rocks

Posts: 5

Re: Problem with multpile (fast) client-requests (Eth)
Reply #13 - 01.04.2009 at 13:28:47
 
Yes, I did, but no luck again. I'll try again sometime available.
Back to top
 
 
View Profile   IP Logged
etracer
Full Member
***
Offline

Arduino rocks

Posts: 233
Indiana
Gender: male
Re: Problem with multpile (fast) client-requests (Eth)
Reply #14 - 01.04.2009 at 13:30:27
 
Quote:
Oh and this patch is meant for the standard Ethernet library, it doesn't work with Ethernet2 yet.

Will this patch find it's way to the official distribution?


I've only done a casual look at the Ethernet2 library, but from what I saw the Client::connect() code looked identical to the "official" library.

I'll be writing up this patch and submitting it to the developer's list. There are a couple of other bugs and enhancements that I'm still looking into.

1. The client can get into an unrecoverable state if the sketch opens connections and then fails to call client.stop() before issuing another client.connet(). After the fourth connect all sockets will be in use preventing any more connections. The insidious part is that there's no way to release the orphaned connections. Easily fixed by having the connect() return a fail result if there is already and active connection.

2. There are some situations where the client.stop() fails to disconnect and leaves an "ESTABLISHED" connection to the server. I'm still researching this.
Back to top
 
 
View Profile   IP Logged
Pages: 1 2