Public Types | Public Member Functions | Static Public Member Functions | Public Attributes | Protected Member Functions | Static Protected Member Functions | Protected Attributes | Private Member Functions | Private Attributes | Static Private Attributes | List of all members
Downloader Class Referenceabstract

#include <Downloader.h>

Inheritance diagram for Downloader:
Inheritance graph
[legend]
Collaboration diagram for Downloader:
Collaboration graph
[legend]

Public Types

using Answer = DownloaderAnswer
 
typedef CbcPointer< AsyncJob > Pointer
 

Public Member Functions

 Downloader (const SBuf &url, const AsyncCallback< Answer > &, const MasterXactionPointer &, unsigned int level=0)
 
 ~Downloader () override
 
void swanSong () override
 
void downloadFinished ()
 delays destruction to protect doCallouts() More...
 
unsigned int nestedLevel () const
 The nested level of Downloader object (downloads inside downloads). More...
 
void handleReply (clientStreamNode *, ClientHttpRequest *, HttpReply *, StoreIOBuffer)
 
bool canBeCalled (AsyncCall &call) const
 whether we can be called More...
 
void callStart (AsyncCall &call)
 
virtual void callEnd ()
 called right after the called job method More...
 
virtual void callException (const std::exception &e)
 called when the job throws during an async call More...
 
void handleStopRequest ()
 process external request to terminate now (i.e. during this async call) More...
 
virtual void * toCbdata ()=0
 

Static Public Member Functions

static void Start (const Pointer &job)
 
static void RegisterWithCacheManager ()
 

Public Attributes

const InstanceId< AsyncJob > id
 job identifier More...
 

Protected Member Functions

bool doneAll () const override
 whether positive goal has been reached More...
 
void start () override
 called by AsyncStart; do not call directly More...
 
void deleteThis (const char *aReason)
 
void mustStop (const char *aReason)
 
bool done () const
 the job is destroyed in callEnd() when done() More...
 
virtual const char * status () const
 internal cleanup; do not call directly More...
 

Static Protected Member Functions

static void ReportAllJobs (StoreEntry *)
 writes a cache manager report about all jobs existing in this worker More...
 

Protected Attributes

const char * stopReason
 reason for forcing done() to be true More...
 
const char * typeName
 kid (leaf) class name, for debugging More...
 
AsyncCall::Pointer inCall
 the asynchronous call being handled, if any More...
 
bool started_ = false
 Start() has finished successfully. More...
 
bool swanSang_ = false
 swanSong() was called More...
 

Private Member Functions

 CBDATA_CHILD (Downloader)
 
bool buildRequest ()
 Initializes and starts the HTTP GET request to the remote server. More...
 
void callBack (Http::StatusCode const status)
 
virtual void finalizedInCbdataChild ()=0
 hack: ensure CBDATA_CHILD() after a toCbdata()-defining CBDATA_INTERMEDIATE() More...
 

Private Attributes

SBuf url_
 the url to download More...
 
AsyncCallback< Answer > callback_
 answer destination More...
 
SBuf object_
 the object body data More...
 
const unsigned int level_
 holds the nested downloads level More...
 
MasterXactionPointer masterXaction_
 download transaction context More...
 
DownloaderContextPointer context_
 Pointer to an object that stores the clientStream required info. More...
 

Static Private Attributes

static const size_t MaxObjectSize = 1*1024*1024
 The maximum allowed object size. More...
 

Detailed Description

The Downloader class fetches SBuf-storable things for other Squid components/transactions using internal requests. For example, it is used to fetch missing intermediate certificates when validating origin server certificate chains.

Definition at line 45 of file Downloader.h.

Member Typedef Documentation

◆ Answer

using Downloader::Answer = DownloaderAnswer

Definition at line 49 of file Downloader.h.

◆ Pointer

typedef CbcPointer<AsyncJob> AsyncJob::Pointer
inherited

Definition at line 34 of file AsyncJob.h.

Constructor & Destructor Documentation

◆ Downloader()

Downloader::Downloader ( const SBuf &  url,
const AsyncCallback< Answer > &  cb,
const MasterXactionPointer &  mx,
unsigned int  level = 0 
)

Definition at line 71 of file Downloader.cc.

◆ ~Downloader()

Downloader::~Downloader ( )
override

Definition at line 80 of file Downloader.cc.

References debugs.

Member Function Documentation

◆ buildRequest()

bool Downloader::buildRequest ( )
private

Definition at line 136 of file Downloader.cc.

References SBuf::c_str(), ClientHttpRequest::calloutContext, ClientHttpRequest::client_stream, clientGetMoreData, clientReplyDetach, clientReplyStatus, clientStreamInit(), context_, StoreIOBuffer::data, Http::DATE, debugs, ClientHttpRequest::doCallouts(), downloaderDetach(), downloaderRecipient(), HttpRequest::FromUrl(), RefCount< C >::getRaw(), Http::HOST, HTTP_REQBUF_SZ, ClientHttpRequest::initRequest(), StoreIOBuffer::length, masterXaction_, Http::METHOD_GET, Http::ProtocolVersion(), ClientHttpRequest::req_sz, DownloaderContext::requestBuffer, squid_curtime, ClientHttpRequest::uri, url_, and xstrdup.

Referenced by start().

◆ callBack()

void Downloader::callBack ( Http::StatusCode const  statusCode)
private

Schedules for execution the "callback" with parameters the status and object.

Definition at line 260 of file Downloader.cc.

References assert, callback_, CallJobHere, downloadFinished(), object_, ScheduleCallHere, and Http::scOkay.

Referenced by handleReply(), start(), and swanSong().

◆ callEnd()

void AsyncJob::callEnd ( )
virtualinherited

called right after the called job method

Reimplemented in Adaptation::Icap::Xaction.

Definition at line 152 of file AsyncJob.cc.

References assert, AsyncCall::debugLevel, debugs, AsyncCall::debugSection, AsyncJob::done(), AsyncJob::inCall, AsyncJob::started_, AsyncJob::status(), AsyncJob::swanSang_, AsyncJob::swanSong(), and AsyncJob::typeName.

Referenced by Adaptation::Icap::Xaction::callEnd(), and AsyncJob::deleteThis().

◆ callException()

void AsyncJob::callException ( const std::exception &  e)
virtualinherited

Reimplemented in Adaptation::Icap::ModXact, Adaptation::Icap::Xaction, Adaptation::Icap::ServiceRep, Ftp::Server, Ipc::Forwarder, Ipc::Inquirer, ConnStateData, ClientHttpRequest, and Rock::Rebuild.

Definition at line 143 of file AsyncJob.cc.

References cbdataReferenceValid(), debugs, Must, AsyncJob::mustStop(), and CbdataParent::toCbdata().

Referenced by Ipc::Inquirer::callException(), Ipc::Forwarder::callException(), Ftp::Server::callException(), Adaptation::Icap::Xaction::callException(), and ConnStateData::callException().

◆ callStart()

void AsyncJob::callStart ( AsyncCall &  call)
inherited

called just before the called method

Definition at line 130 of file AsyncJob.cc.

References cbdataReferenceValid(), AsyncCall::debugLevel, debugs, AsyncCall::debugSection, AsyncJob::inCall, Must, AsyncJob::status(), CbdataParent::toCbdata(), and AsyncJob::typeName.

◆ canBeCalled()

bool AsyncJob::canBeCalled ( AsyncCall &  call) const
inherited

Definition at line 117 of file AsyncJob.cc.

References AsyncCall::cancel(), debugs, and AsyncJob::inCall.

◆ CBDATA_CHILD()

Downloader::CBDATA_CHILD ( Downloader  )
private

◆ deleteThis()

void AsyncJob::deleteThis ( const char *  aReason)
protectedinherited

Definition at line 65 of file AsyncJob.cc.

References asyncCall(), AsyncJob::callEnd(), debugs, AsyncJob::inCall, JobMemFun(), Must, AsyncJob::stopReason, and AsyncJob::typeName.

Referenced by ConnStateData::connStateClosed().

◆ done()

bool AsyncJob::done ( ) const
protectedinherited

Definition at line 106 of file AsyncJob.cc.

References AsyncJob::doneAll(), and AsyncJob::stopReason.

Referenced by AsyncJob::callEnd(), HappyConnOpener::checkForNewConnection(), downloadFinished(), and HappyConnOpener::maybeOpenPrimeConnection().

◆ doneAll()

bool Downloader::doneAll ( ) const
overrideprotectedvirtual

Reimplemented from AsyncJob.

Definition at line 100 of file Downloader.cc.

References callback_, and AsyncJob::doneAll().

◆ downloadFinished()

void Downloader::downloadFinished ( )

Definition at line 251 of file Downloader.cc.

References debugs, AsyncJob::done(), and Must.

Referenced by callBack().

◆ finalizedInCbdataChild()

virtual void CbdataParent::finalizedInCbdataChild ( )
privatepure virtualinherited

◆ handleReply()

void Downloader::handleReply ( clientStreamNode *  node,
ClientHttpRequest *  http,
HttpReply *  reply,
StoreIOBuffer  receivedData 
)

Definition at line 192 of file Downloader.cc.

References SBuf::append(), assert, callBack(), clientStreamRead(), clientStreamStatus(), Http::Message::content_length, context_, StoreIOBuffer::data, debugs, StoreIOBuffer::error, fatal(), StoreIOBuffer::flags, HTTP_REQBUF_SZ, StoreIOBuffer::length, SBuf::length(), MaxObjectSize, SBuf::maxSize, object_, StoreIOBuffer::offset, ClientHttpRequest::Out::offset, ClientHttpRequest::out, DownloaderContext::requestBuffer, Http::scInternalServerError, Http::scOkay, ClientHttpRequest::Out::size, STREAM_COMPLETE, STREAM_FAILED, STREAM_NONE, and STREAM_UNPLANNED_COMPLETE.

Referenced by downloaderRecipient().

◆ handleStopRequest()

void AsyncJob::handleStopRequest ( )
inlineinherited

Definition at line 73 of file AsyncJob.h.

References AsyncJob::mustStop().

◆ mustStop()

void AsyncJob::mustStop ( const char *  aReason)
protectedinherited

Definition at line 85 of file AsyncJob.cc.

References debugs, AsyncJob::inCall, Must, AsyncJob::stopReason, and AsyncJob::typeName.

Referenced by HttpStateData::abortAll(), AsyncJob::callException(), HttpStateData::continueAfterParsingHeader(), HttpStateData::drop1xx(), HttpStateData::handleMoreRequestBodyAvailable(), AsyncJob::handleStopRequest(), HttpStateData::httpStateConnClosed(), HttpStateData::httpTimeout(), HttpStateData::proceedAfter1xx(), ConnStateData::proxyProtocolError(), HttpStateData::readReply(), HttpStateData::start(), and HttpStateData::wroteLast().

◆ nestedLevel()

unsigned int Downloader::nestedLevel ( ) const
inline

Definition at line 59 of file Downloader.h.

References level_.

◆ RegisterWithCacheManager()

void AsyncJob::RegisterWithCacheManager ( )
staticinherited

Definition at line 215 of file AsyncJob.cc.

References Mgr::RegisterAction(), and AsyncJob::ReportAllJobs().

Referenced by mainInitialize().

◆ ReportAllJobs()

void AsyncJob::ReportAllJobs ( StoreEntry *  e)
staticprotectedinherited

Definition at line 198 of file AsyncJob.cc.

References AllJobs().

Referenced by AsyncJob::RegisterWithCacheManager().

◆ start()

void Downloader::start ( )
overrideprotectedvirtual

Reimplemented from AsyncJob.

Definition at line 185 of file Downloader.cc.

References buildRequest(), callBack(), and Http::scInternalServerError.

◆ Start()

void AsyncJob::Start ( const Pointer &  job)
staticinherited

Promises to start the configured job (eventually). The job is deemed to be running asynchronously beyond this point, so the caller should only access the job object via AsyncCalls rather than directly.

swanSong() is only called for jobs for which this method has returned successfully (i.e. without throwing).

Definition at line 37 of file AsyncJob.cc.

References CallJobHere, AsyncJob::start(), and AsyncJob::started_.

Referenced by Ftp::Server::AcceptCtrlConnection(), clientListenerConnectionOpened(), Ipc::Coordinator::handleCacheMgrRequest(), Ipc::Coordinator::handleSnmpRequest(), httpAccept(), httpsAccept(), httpStart(), idnsInitVC(), Ftp::Gateway::listenForDataChannel(), Ftp::Server::listenForDataConnection(), Log::TcpLogger::Open(), peerProbeConnect(), Mgr::FunAction::respond(), Mgr::InfoAction::respond(), Ipc::SendMessage(), Mgr::Inquirer::sendResponse(), snmpConstructReponse(), SquidMain(), Adaptation::AccessCheck::Start(), CacheManager::start(), Rock::Rebuild::Start(), JobWaitBase::start_(), BodyPipe::startAutoConsumptionIfNeeded(), Ftp::StartGateway(), Ftp::StartRelay(), and Rock::SwapDir::updateHeaders().

◆ status()

const char * AsyncJob::status ( ) const
protectedvirtualinherited

for debugging, starts with space

Reimplemented in Adaptation::Icap::ServiceRep, HappyConnOpener, Adaptation::Icap::Xaction, Adaptation::Ecap::XactionRep, Security::PeerConnector, Http::Tunneler, Adaptation::Initiate, Comm::TcpAcceptor, and Ipc::Inquirer.

Definition at line 182 of file AsyncJob.cc.

References MemBuf::append(), Packable::appendf(), MemBuf::content(), MemBuf::reset(), AsyncJob::stopReason, and MemBuf::terminate().

Referenced by AsyncJob::callEnd(), AsyncJob::callStart(), Comm::TcpAcceptor::status(), and Adaptation::Initiate::status().

◆ swanSong()

void Downloader::swanSong ( )
overridevirtual

Reimplemented from AsyncJob.

Definition at line 86 of file Downloader.cc.

References callBack(), callback_, context_, debugs, DownloaderContext::finished(), and Http::scInternalServerError.

◆ toCbdata()

virtual void* CbdataParent::toCbdata ( )
pure virtualinherited

Referenced by AsyncJob::callException(), and AsyncJob::callStart().

Member Data Documentation

◆ callback_

AsyncCallback<Answer> Downloader::callback_
private

Definition at line 80 of file Downloader.h.

Referenced by callBack(), doneAll(), and swanSong().

◆ context_

DownloaderContextPointer Downloader::context_
private

Definition at line 87 of file Downloader.h.

Referenced by buildRequest(), handleReply(), and swanSong().

◆ id

const InstanceId<AsyncJob> AsyncJob::id
inherited

Definition at line 75 of file AsyncJob.h.

◆ inCall

AsyncCall::Pointer AsyncJob::inCall
protectedinherited

Definition at line 86 of file AsyncJob.h.

Referenced by AsyncJob::callEnd(), AsyncJob::callStart(), AsyncJob::canBeCalled(), AsyncJob::deleteThis(), and AsyncJob::mustStop().

◆ level_

const unsigned int Downloader::level_
private

Definition at line 83 of file Downloader.h.

Referenced by nestedLevel().

◆ masterXaction_

MasterXactionPointer Downloader::masterXaction_
private

Definition at line 84 of file Downloader.h.

Referenced by buildRequest().

◆ MaxObjectSize

const size_t Downloader::MaxObjectSize = 1*1024*1024
staticprivate

Definition at line 75 of file Downloader.h.

Referenced by handleReply().

◆ object_

SBuf Downloader::object_
private

Definition at line 82 of file Downloader.h.

Referenced by callBack(), and handleReply().

◆ started_

bool AsyncJob::started_ = false
protectedinherited

Definition at line 88 of file AsyncJob.h.

Referenced by AsyncJob::callEnd(), AsyncJob::Start(), and AsyncJob::~AsyncJob().

◆ stopReason

const char* AsyncJob::stopReason
protectedinherited

Definition at line 84 of file AsyncJob.h.

Referenced by AsyncJob::deleteThis(), AsyncJob::done(), AsyncJob::mustStop(), AsyncJob::status(), and HappyConnOpener::status().

◆ swanSang_

bool AsyncJob::swanSang_ = false
protectedinherited

Definition at line 89 of file AsyncJob.h.

Referenced by AsyncJob::callEnd(), and AsyncJob::~AsyncJob().

◆ typeName

const char* AsyncJob::typeName
protectedinherited

Definition at line 85 of file AsyncJob.h.

Referenced by AsyncJob::AsyncJob(), AsyncJob::callEnd(), AsyncJob::callStart(), AsyncJob::deleteThis(), AsyncJob::mustStop(), Adaptation::Icap::Xaction::Xaction(), and AsyncJob::~AsyncJob().

◆ url_

SBuf Downloader::url_
private

Definition at line 77 of file Downloader.h.

Referenced by buildRequest().

The documentation for this class was generated from the following files:

 

Introduction

Documentation

Support

Miscellaneous