From eliezer at ngtech.co.il  Wed Oct 16 00:28:35 2013
From: eliezer at ngtech.co.il (Eliezer Croitoru)
Date: Wed, 16 Oct 2013 03:28:35 +0300
Subject: Using eCAP to black\white\gray list clients?
Message-ID: <525DDDB3.2060205@ngtech.co.il>

I am wondering about how to use eCAP to make some users understand that 
their IP is abusing the service..

In squid I can mark connections and some other stuff while the OS does 
the actuall limit.
I have tried to acutally read the eCAP examples etc but never understood 
how to really make use of it.
I know that there are docs but I am not starting reading them over again.

So what would be the best code implementation that would run this 
function?, Block\White\Gray list users by IP?
In ruby I can use very simple libraries and very clean code to implenent 
the DB on REDIS\TOKYOTYRANT\MEMCACHED etc.

Issues with eCAP:
- What aspect do I need to take about concurrent requests handling?
- Can I use standard libs for MYSQL or any other DB?
- what would you choose?

Please take in considiration that I know a bit c a bit c++ 8 bits of 
Ruby 1-2 bits of perl and snippets of py php and some others.

Thanks,
Eliezer


From rousskov at measurement-factory.com  Wed Oct 16 00:48:04 2013
From: rousskov at measurement-factory.com (Alex Rousskov)
Date: Tue, 15 Oct 2013 18:48:04 -0600
Subject: Mailing list fixed
Message-ID: <525DE244.6000708@measurement-factory.com>

Hello,

    Just wanted to let you know that if you tried to post a message to
this mailing list in the past, it was NOT received by the list
management software due to a server misconfiguration. Web-based list
subscriptions worked fine, but no email could get through.

Feel free to repost your message if it is still relevant, of course. You
should at least get a "your message awaits moderation" auto-response
this time.


I apologize for the inconvenience and would like to thank Eliezer
Croitoru for helping us triage the problem.


Alex.


From rousskov at measurement-factory.com  Wed Oct 16 03:41:17 2013
From: rousskov at measurement-factory.com (Alex Rousskov)
Date: Tue, 15 Oct 2013 21:41:17 -0600
Subject: Using eCAP to black\white\gray list clients?
In-Reply-To: <525DDDB3.2060205@ngtech.co.il>
References: <525DDDB3.2060205@ngtech.co.il>
Message-ID: <525E0ADD.7050303@measurement-factory.com>

On 10/15/2013 06:28 PM, Eliezer Croitoru wrote:

> I am wondering about how to use eCAP to make some users understand that
> their IP is abusing the service..


Hello Eliezer,

    The easiest route may be to write a REQMOD adapter that calls
host::Xaction::blockVirgin() when (or after) the abuse is detected. That
approach relies on the host application ability to block messages. For
example, Squid supports that and offers some flexibility via deny_info
configuration IIRC.

If you need direct control over blocking messages from the adapter
itself, then a REQMOD adapter can perform "request satisfaction" (i.e,
respond to a virgin request with a custom reply) by calling
host::Xaction::useAdapted(reply) instead of calling blockVirgin().


> In squid I can mark connections and some other stuff while the OS does
> the actuall limit.
> I have tried to acutally read the eCAP examples etc but never understood
> how to really make use of it.
> I know that there are docs but I am not starting reading them over again.

Well, if you want an adapter, but do not want to study the samples and
the primary libecap interfaces (that the samples use), you would need to
find somebody willing to do that work for you. I cannot think of a third
alternative.


> So what would be the best code implementation that would run this
> function?, Block\White\Gray list users by IP?
> In ruby I can use very simple libraries and very clean code to implenent
> the DB on REDIS\TOKYOTYRANT\MEMCACHED etc.

An eCAP adapter may use existing C and C++ libraries as well, of course.
Many adapters use libraries. For example, the eCAP ClamAV adapter
available at e-cap.org is using the ClamAV library.

If you must use Ruby, you would need to wrap Ruby calls into a C++ shim.
I do not know how difficult developing a good Ruby shim would be. I have
seen a Python (IIRC) shim prototype a few years ago. It did not look
incredibly complex, but it did require specialized knowledge of how to
bridge the two languages. I would not recommend attempting to build one
unless Ruby makes that task incredibly easy or you know what you are doing.


> Issues with eCAP:

> - What aspect do I need to take about concurrent requests handling?

At any given time, multiple adapter::Xaction objects may coexist, but at
most one object method may be called by the host application. There is
no true concurrency and associated headaches unless you want to create
threads in your adapter. Threaded adapters are supported since libecap
v1.0 and are discussed in docs/async.txt.


> - Can I use standard libs for MYSQL or any other DB?

Without adapter threads, you can use any C or C++ library that has a
blocking API. With adapter threads, the library may be asynchronous as
well (and your adapter will be responsible for synchronizing library and
host actions as needed).


> - what would you choose?

I would build a synchronous prototype using std::map or a similar
in-memory non-shared "database". If the prototype works well, I would
upgrade to a real database that can share state across multiple host
application processes and preserve state across host application
crashes. Sorry, I do not have a specific recommendation, but I am sure
there are a few candidates to pick from.


> Please take in considiration that I know a bit c a bit c++ 8 bits of
> Ruby 1-2 bits of perl and snippets of py php and some others.

Sounds like a good base to study available eCAP adapters and modify one
of them into a prototype you need :-).


Good luck,

Alex.