New look in the squid web.

Adrian Chadd is an active squid developer who has helped with the new look on the web page.



The squid team is remarking the new "How to Help out" section in order to catch more people helping to improve the squid software.

It seems in a near future We'll have a new merchandise section where the people from the list could buy squid related things like t-shirts, cup of tea or similar in order to help the project.

Personally I feel the squid list is a good source of information so We all would have to feel more close to the squid and to try help to the squid team by some way . Because YOU can help in a differentd ways.

SSL support, Squid 2.6 branch and RedHat

Testing the last squid version (squid-2.6Stable9) with ssl support in order to operate as a reverse proxy, I get this errors in the compilation process:

---cut---
ssl_support.h:49: syntax error before '*' token
ssl_support.h:49: warning: type defaults to `int' in declaration of `sslCreateServerContext'
ssl_support.h:49: warning: data definition has no type or storage class
ssl_support.h:50: syntax error before '*' token
ssl_support.h:50: warning: type defaults to `int' in declaration of `sslCreateClientContext'
ssl_support.h:50: warning: data definition has no type or storage class
ssl_support.h:54: syntax error before "SSL"
ssl_support.h:56: syntax error before '*' token
ssl_support.h:57: syntax error before '*' token
ssl_support.h:58: syntax error before '*' token
ssl_support.h:59: syntax error before '*' token
ssl_support.h:60: syntax error before '*' token
---cut--

What's happening?

The problem here is with RedHat, They have built OpenSSL with Kerberos support

#[root@proxy squid-2.6.STABLE9]# rpm -qR openssl-devel-0.9.7a-33.12
krb5-devel

but Kerberos is not in the standard library and include path. This makes it impossible to build OpenSSL applications without manually including /usr/kerberos in the include and library paths.

More info.

Reverse proxy configurations

The configuration of a reverse proxy, it depends on what functionality you want to achive.
There is three ways of using this depending on what your functionality
requirements are:

a) With Squid acting as an accelerator/reverse proxy for a defined list
of sites, upgrading these sites to https. You then use the ssl option to
cache_peer to wrap the request in SSL.

b) By using a HTTP client sending https:// URLs to Squid. Squid will
then maintain the SSL on behalf of the client.

Here, the client has to send the https:// request using HTTP to the
proxy, just as it does for http://. This is:

GET https://www.example.com/path/to/file HTTP/1.1
[headers]

It does not work for clients using the CONNECT method asking for a SSL
tunnel over the proxy.

At this case, the clients are knowing they should not run the SSL themselves and
delegating this task to the proxy. They don't have any SSL capabilities and instead
rely on the proxy to perform the SSL encryption.


c) Using a url rewriter helper to rewrite selected http:// URLs into
https:// per your own specifications, making Squid process the request
as a https:// request even if the client requested http://

At this case, the clients are emulating this by rewriting http:// URLs into https://
at the proxy.

It's also possible to extend Squid with the capability to decrypt
CONNECT SSL proxy requests allowing inspection of https traffic.
For more information on this way you can contact with Henrik Nordstrom.
Contactos de squid.

Squid running out of free ports.

Symptons from a busy squid with high traffic:

commBind: Cannot bind socket FD 98 to *:0: (98) Address already in use

Solution:

You have run out of free ports, all available ports occupied by
TIME_WAIT sockets.

Things to look into

1. Make sure you internally use persistent connections between Squid and
the web servers. This cuts down on the number of initiated connections/s
considerably.

2. Configure the unassigned port range as big as possible in your OS. On
Linux this is set in /proc/sys/net/ipv4/ip_local_port_range. The biggest
possible range is 1024-65535 and can sustain up to at least 500
connections/s continuous load squid->webservers.

What does Squid do or act like when its out of file descriptors?

When Squid sees it's short of filedescriptors it stops accepting new
requests, focusing on finishing what it has already accepted.

And long before there is a shortage it disables the use of persistent
connections to limit the pressure on concurrent filedescriptors.

What does it to do in such case?

Once Squid has detected a filedescriptor limitation it won't go
above the number of filedescriptor it used at that time, and you need to
restart Squid to recover after fixing the cause to the system wide
filedescriptor shortage.

do squid recover or do it need to be restarted?

depends on the reason to the filedescriptor shortage.

If the shortage is due to Squid using very many filedescriptors then no
action need to be taken (except perhaps increase the amount of
filedescriptors available to Squid to avoid the problem in future).
Squid automatically adjusts to the per process limit and hitting the
system wide limit if it's lower than the per-process limit.

If the shortage is due to some other process causing the systems as a
whole to temporarily run short of filedescriptors or related resources
then you need to restart Squid after fixing the problem as Squid has got
fooled in this situation into thinking that your system can not support
a reasonable amount of active connections.

Different log for each domain

With squid 2.6 in a reverse proxy configuration with several domains, it's possible to have log messagees to separate files per cache_peer_domain.

Example with two peer domains.
www.abc.com
www.xyz.com

---squid.conf----
acl abc dstdomain www.abc.com
acl xyz dstdomain www.xyz.com

access_log /path/to/xyz.log squid xyz
access_log none xyz
access_log /path/to/abc.log squid abc
access_log none abc
access_log /path/to/access.log squid
---squid.conf end---

HTTP1.0 / HTTP1.1

Squid is a HTTP/1.0 proxy. But all HTTP/1.1 clients and servers MUST interoperate with HTTP/1.0 to be compliant.

The main difference this brings to the protocol (apart from version number) is that aplications or servers can't use transfer-encoding. The rest of HTTP/1.1 is supported fine over HTTP/1.0.

Transfer-Encoding tells the receiver what encoding has been performed on the message in order for it to be safely transported.
All transfer-encoding values are case-insensitive. HTTP/1.1 uses transfer-encoding values in the TE header field and in the Transfer-Encoding header field. The latest HTTP specification defines only one transfer encoding, chunked encoding.

Relay server for SSL connections

If you have overladed web servers you can save the SSL load put in front of their a relay proxy for ssl connections. This means that this proxy will accept HTTPS connections and will translate them into HTTP requests on another server.

This can be achieve with:
https_port directive.
and setting up squid as an https accelerator for the http server.

Example configuration;
https_port 443 cert=/path/to/sslcert.pem defaultsite=your.site.name
cache_peer ip.of.apache 80 0 no-query originserver

Redirector in a reverse proxy scenario

A common technique using in a reverse proxy scenario where virtual hostnames are distributed to several internal servers is using a redirector. But with the new 2.6 squid there is a better way to achieve this feature.
You can forward the requests with:
cache_peer + cache_peer_acess + never_direct

This is the recommend method if the other server is supposed to act exactly like the real web site, including it's name.

2Gb access.log

"FATAL: logfileWrite: /var/log/squid/access.log: (11) Resource
> temporarily unavailable"

A quit common cause for this message is the access.log reaching the magic 2GB barrier of 32-bit applications.

A common solution for this is compile squid with large cache files support.

From the configure script.

--with-large-files Enable support for large files (logs etc).
--enable-large-cache-files
Enable support for large cache files (>2GB).
WARNING: on-disk cache format is changed by this option

Use the no_cache directive correctly

If you don't want cache some object, the best directive to achieve it is the
no_cache directive

acl QUERY urlpath_regex cgi-bin \? \.asp$ \.php$
no_cache deny QUERY

The refresh_pattern directive only applies on pages where there is no explicit expiry information set by the origin server.

Limit the amount of time spending in personal stuff during work hours

Several people in her works hours are doing personal stuff such as reading gmail, yahoo mail, reading blogs, etc. In order to avoid these kind of behaviour you can do some acctions to remember them they are working and it's not good wasting work hours with not related things.

Help to accomplish this task:

session helper from squid-2.6.
deny_info redirecting to a web page with a reminder.

Recommended compile options

--prefix= if you want to install Squid in another location
than /usr/local/squid

--enable-storeio="ufs aufs null" on most platforms to allow choice of
more performing disk I/O.

--enable-auth to enable all authentication schemes to have them
available the day you want to use them.

--enable-delay-pools to enable the delay pools shaping capability.

--enable-snmp to enable monitoring & statistics collection using SNMP

--with-large-files if you are on a 32-bit platform to enable support
for large log files bigger than 2GB. Warning: Do not use this option on
64-bit platforms.

Golden Rule

Start with the first error, ignore the rest.

Some notes regarding NTLM

Squid 2.6Stable 3 now support the NTLM passthrough.

An alternative which is recommended and works for all proxies is to have the web site using https on authenticated content. https is tunneled via the proxy, not proxied, and therefore works fine even with non-HTTP-compliant authentication such as NTLM.

From now this blog will be in english language

Aumentar numero de Filedescriptores

Cuando compilamos una versión de squid, tenemos que tener en cuenta el número de filedescriptors que va a tener disponible. Cuando realizamos el configure, aparecerá una linea;

checking Maximum number of filedescriptors we can open... 1024

que es los que actualmente tenemos en nuestro sistema operativo disponibles.
Para aumentarlos;

#ulimit -HSn 2048

Volvemos a compilar, y comprobamos que efectivamente ha cogido bien el soporte de filedescriptors que ahora tendrá disponible.

checking Maximum number of filedescriptors we can open... 2048

1024 Filedescriptors








2048 Filedescriptors

SNMP y Squid

Para ver los diferentes parametros que podemos monitorizar con squid en mrtg.

snmpwalk host:3401 -v 1 -c public .1.3.6.1.4.1.3495 -m /usr/local/squid/share/mib.txt

Para lo que previamente tendremos que haber configurado el acceso snmp en squid.conf
para que permita consultas a host.

Ocultar la versión de squid

Para ocultar la versión de squid que se muestra en una página de error;

Edita src/errorpage.c
Linea:69 :)

>De
"Generated %T by %h (%s)\n"
a
"Generated %T by %h \n"

Y vuelve a compilar squid.

Update on 6th September
From 2.6Stable1 it's available the httpd_supress_version_string directive (default off)

Bloquear Malware con Squid

Hoy en día existen URLS que ocultan todo tipo de efectos dañiños tales como Virus, troyanos gusanos y todo lo denominado malware.
Un usuario desea saber como se puede conseguir con squid filtrar archivos peligrosos por su extensión, pero las típicas reglas que bloquean
\.com$
\.scr$
\.bat$
estas extensiones no sirven cuando hoy en día existen multitud de técnicas para ocultar archivos en urls tales como
http://www.mikes.educv.ro/albums/cartao.scr?4d325356ae47122a6e7b8f1f07cae26d
La solución para esto pasa por integrar squid con listas de URLs disponibles para él.
Aquí se explica la configuración de squid.