Package org.eclipse.jgit.transport

Class UploadPack

  • public class UploadPack
extends java.lang.Object
    Implements the server side of a fetch connection, transmitting objects.

    • Field Detail

      • db

        private final Repository db
        Database we read the objects from.

      • walk

        private final RevWalk walk
        Revision traversal support over db.

      • packConfig

        private PackConfig packConfig
        Configuration to pass into the PackWriter.

      • transferConfig

        private TransferConfig transferConfig
        Configuration for various transfer options.

      • timeout

        private int timeout
        Timeout in seconds to wait for client interaction.

      • biDirectionalPipe

        private boolean biDirectionalPipe
        Is the client connection a bi-directional socket or pipe?

        If true, this class assumes it can perform multiple read and write cycles with the client over the input and output streams. This matches the functionality available with a standard TCP/IP connection, or a local operating system or in-memory pipe.

        If false, this class runs in a read everything then output results mode, making it suitable for single round-trip systems RPCs such as HTTP.

      • clientRequestedV2

        private boolean clientRequestedV2
        Whether the client requested to use protocol V2 through a side channel (such as the Git-Protocol HTTP header).

      • rawIn

        private java.io.InputStream rawIn

      • msgOut

        private java.io.OutputStream msgOut

      • protocolV2Hook

        private ProtocolV2Hook protocolV2Hook
        Hook used while processing Git protocol v2 requests.

      • advertiseRefsHook

        private AdvertiseRefsHook advertiseRefsHook
        Hook used while advertising the refs to the client.

      • advertiseRefsHookCalled

        private boolean advertiseRefsHookCalled
        Whether the advertiseRefsHook has been invoked.

      • refFilter

        private RefFilter refFilter
        Filter used while advertising the refs to the client.

      • preUploadHook

        private PreUploadHook preUploadHook
        Hook handling the various upload phases.

      • postUploadHook

        private PostUploadHook postUploadHook
        Hook for taking post upload actions.

      • userAgent

        java.lang.String userAgent
        Caller user agent

      • wantIds

        private java.util.Set<ObjectId> wantIds
        Raw ObjectIds the client has asked for, before validating them.

      • wantAll

        private final java.util.Set<RevObject> wantAll
        Objects the client wants to obtain.

      • commonBase

        private final java.util.Set<RevObject> commonBase
        Objects on both sides, these don't have to be sent.

      • oldestTime

        private int oldestTime
        Commit time of the oldest common commit, in seconds.

      • okToGiveUp

        private java.lang.Boolean okToGiveUp
        null if commonBase should be examined again.

      • sentReady

        private boolean sentReady

      • advertised

        private java.util.Set<ObjectId> advertised
        Objects we sent in our advertisement list.

      • WANT

        private final RevFlag WANT
        Marked on objects the client has asked us to give them.

      • PEER_HAS

        private final RevFlag PEER_HAS
        Marked on objects both we and the client have.

      • SATISFIED

        private final RevFlag SATISFIED
        Objects where we found a path from the want list to a common base.

      • noDone

        private boolean noDone

      • currentRequest

        private FetchRequest currentRequest
        Request this instance is handling. We need to keep a reference to it for pre upload hooks. They receive a reference this instance and invoke methods like getDepth() to get information about the request.

    • Constructor Detail

      • UploadPack

        public UploadPack​(Repository copyFrom)
        Create a new pack upload for an open repository.
        Parameters:
        copyFrom - the source repository.

    • Method Detail

      • getRepository

        public final Repository getRepository()
        Get the repository this upload is reading from.
        Returns:
        the repository this upload is reading from.

      • getRevWalk

        public final RevWalk getRevWalk()
        Get the RevWalk instance used by this connection.
        Returns:
        the RevWalk instance used by this connection.

      • getAdvertisedRefs

        public final java.util.Map<java.lang.String,​Ref> getAdvertisedRefs()
        Get refs which were advertised to the client.
        Returns:
        all refs which were advertised to the client. Only valid during the negotiation phase. Will return null if setAdvertisedRefs(Map) has not been called yet or if #sendPack() has been called.

      • setAdvertisedRefs

        public void setAdvertisedRefs​(@Nullable
                              java.util.Map<java.lang.String,​Ref> allRefs)
        Set the refs advertised by this UploadPack.

        Intended to be called from a PreUploadHook.

        Parameters:
        allRefs - explicit set of references to claim as advertised by this UploadPack instance. This overrides any references that may exist in the source repository. The map is passed to the configured getRefFilter(). If null, assumes all refs were advertised.

      • getTimeout

        public int getTimeout()
        Get timeout (in seconds) before aborting an IO operation.
        Returns:
        timeout (in seconds) before aborting an IO operation.

      • setTimeout

        public void setTimeout​(int seconds)
        Set the timeout before willing to abort an IO call.
        Parameters:
        seconds - number of seconds to wait (with no data transfer occurring) before aborting an IO read or write operation with the connected client.

      • isBiDirectionalPipe

        public boolean isBiDirectionalPipe()
        Whether this class expects a bi-directional pipe opened between the client and itself.
        Returns:
        true if this class expects a bi-directional pipe opened between the client and itself. The default is true.

      • setBiDirectionalPipe

        public void setBiDirectionalPipe​(boolean twoWay)
        Set whether this class will assume the socket is a fully bidirectional pipe between the two peers
        Parameters:
        twoWay - if true, this class will assume the socket is a fully bidirectional pipe between the two peers and takes advantage of that by first transmitting the known refs, then waiting to read commands. If false, this class assumes it must read the commands before writing output and does not perform the initial advertising.

      • getRequestPolicy

        public UploadPack.RequestPolicy getRequestPolicy()
        Get policy used by the service to validate client requests
        Returns:
        policy used by the service to validate client requests, or null for a custom request validator.

      • setRequestValidator

        public void setRequestValidator​(@Nullable
                                UploadPack.RequestValidator validator)
        Set custom validator for client want list.
        Parameters:
        validator - custom validator for client want list.
        Since:
        3.1

      • getAdvertiseRefsHook

        public AdvertiseRefsHook getAdvertiseRefsHook()
        Get the hook used while advertising the refs to the client.
        Returns:
        the hook used while advertising the refs to the client.

      • getRefFilter

        public RefFilter getRefFilter()
        Get the filter used while advertising the refs to the client.
        Returns:
        the filter used while advertising the refs to the client.

      • setAdvertiseRefsHook

        public void setAdvertiseRefsHook​(@Nullable
                                 AdvertiseRefsHook advertiseRefsHook)
        Set the hook used while advertising the refs to the client.

        If the AdvertiseRefsHook chooses to call setAdvertisedRefs(Map), only refs set by this hook and selected by the RefFilter will be shown to the client.

        Parameters:
        advertiseRefsHook - the hook; may be null to show all refs.

      • setProtocolV2Hook

        public void setProtocolV2Hook​(@Nullable
                              ProtocolV2Hook hook)
        Set the protocol V2 hook.
        Parameters:
        hook - the hook; if null no special actions are taken.
        Since:
        5.1

      • getProtocolV2Hook

        public ProtocolV2Hook getProtocolV2Hook()
        Get the currently installed protocol v2 hook.
        Returns:
        the hook or a default implementation if none installed.
        Since:
        5.5

      • setRefFilter

        public void setRefFilter​(@Nullable
                         RefFilter refFilter)
        Set the filter used while advertising the refs to the client.

        Only refs allowed by this filter will be sent to the client. The filter is run against the refs specified by the AdvertiseRefsHook (if applicable). If null or not set, uses the filter implied by the TransferConfig.

        Parameters:
        refFilter - the filter; may be null to show all refs.

      • getPreUploadHook

        public PreUploadHook getPreUploadHook()
        Get the configured pre upload hook.
        Returns:
        the configured pre upload hook.

      • setPreUploadHook

        public void setPreUploadHook​(@Nullable
                             PreUploadHook hook)
        Set the hook that controls how this instance will behave.
        Parameters:
        hook - the hook; if null no special actions are taken.

      • getPostUploadHook

        public PostUploadHook getPostUploadHook()
        Get the configured post upload hook.
        Returns:
        the configured post upload hook.
        Since:
        4.1

      • setPostUploadHook

        public void setPostUploadHook​(@Nullable
                              PostUploadHook hook)
        Set the hook for post upload actions (logging, repacking).
        Parameters:
        hook - the hook; if null no special actions are taken.
        Since:
        4.1

      • setPackConfig

        public void setPackConfig​(@Nullable
                          PackConfig pc)
        Set the configuration used by the pack generator.
        Parameters:
        pc - configuration controlling packing parameters. If null the source repository's settings will be used.

      • setTransferConfig

        public void setTransferConfig​(@Nullable
                              TransferConfig tc)
        Set configuration controlling transfer options.
        Parameters:
        tc - configuration controlling transfer options. If null the source repository's settings will be used.
        Since:
        3.1

      • isSideBand

        public boolean isSideBand()
                   throws RequestNotYetReadException
        Check whether the client expects a side-band stream.
        Returns:
        true if the client has advertised a side-band capability, false otherwise.
        Throws:
        RequestNotYetReadException - if the client's request has not yet been read from the wire, so we do not know if they expect side-band. Note that the client may have already written the request, it just has not been read.

      • setExtraParameters

        public void setExtraParameters​(java.util.Collection<java.lang.String> params)
        Set the Extra Parameters provided by the client.

        These are parameters passed by the client through a side channel such as the Git-Protocol HTTP header, to allow a client to request a newer response format while remaining compatible with older servers that do not understand different request formats.

        Parameters:
        params - parameters supplied by the client, split at colons or NUL bytes.
        Since:
        5.0

      • setCachedPackUriProvider

        public void setCachedPackUriProvider​(@Nullable
                                     CachedPackUriProvider p)
        Parameters:
        p - provider of URIs corresponding to cached packs (to support the packfile URIs feature)
        Since:
        5.5

      • useProtocolV2

        private boolean useProtocolV2()

      • uploadWithExceptionPropagation

        public void uploadWithExceptionPropagation​(java.io.InputStream input,
                                           java.io.OutputStream output,
                                           @Nullable
                                           java.io.OutputStream messages)
                                    throws ServiceMayNotContinueException,
                                           java.io.IOException
        Execute the upload task on the socket.

        If the client passed extra parameters (e.g., "version=2") through a side channel, the caller must call setExtraParameters first to supply them.

        Parameters:
        input - raw input to read client commands from. Caller must ensure the input is buffered, otherwise read performance may suffer.
        output - response back to the Git network client, to write the pack data onto. Caller must ensure the output is buffered, otherwise write performance may suffer.
        messages - secondary "notice" channel to send additional messages out through. When run over SSH this should be tied back to the standard error channel of the command execution. For most other network connections this should be null.
        Throws:
        ServiceMayNotContinueException - thrown if one of the hooks throws this.
        java.io.IOException - thrown if the server or the client I/O fails, or there's an internal server error.
        Since:
        5.6

      • getStatistics

        public PackStatistics getStatistics()
        Get the PackWriter's statistics if a pack was sent to the client.
        Returns:
        statistics about pack output, if a pack was sent. Null if no pack was sent, such as during the negotiation phase of a smart HTTP connection, or if the client was already up-to-date.
        Since:
        4.1

      • getAdvertisedOrDefaultRefs

        private java.util.Map<java.lang.String,​Ref> getAdvertisedOrDefaultRefs()
                                                                      throws java.io.IOException
        Throws:
        java.io.IOException

      • getFilteredRefs

        private java.util.Map<java.lang.String,​Ref> getFilteredRefs​(java.util.Collection<java.lang.String> refPrefixes)
                                                           throws java.io.IOException
        Throws:
        java.io.IOException

      • mapRefs

        @NonNull
private static java.util.Map<java.lang.String,​Ref> mapRefs​(java.util.Map<java.lang.String,​Ref> refs,
                                                                 java.util.List<java.lang.String> names)
        Returns the specified references.

        This produces an immutable map containing whatever subset of the refs named by the caller are present in the supplied refs map.

        Parameters:
        refs - Map to search for refs to return.
        names - which refs to search for in refs.
        Returns:
        the requested Refs, omitting any that are null or missing.

      • exactRefs

        @NonNull
private java.util.Map<java.lang.String,​Ref> exactRefs​(java.util.List<java.lang.String> names)
                                                     throws java.io.IOException
        Read refs on behalf of the client.

        This checks whether the refs are present in the ref advertisement since otherwise the client might not be supposed to be able to read them.

        Parameters:
        names - unabbreviated names of references.
        Returns:
        the requested Refs, omitting any that are not visible or do not exist.
        Throws:
        java.io.IOException - on failure to read a ref or check it for visibility.

      • findRef

        @Nullable
private Ref findRef​(java.lang.String name)
             throws java.io.IOException
        Find a ref in the usual search path on behalf of the client.

        This checks that the ref is present in the ref advertisement since otherwise the client might not be supposed to be able to read it.

        Parameters:
        name - short name of the ref to find, e.g. "master" to find "refs/heads/master".
        Returns:
        the requested Ref, or null if it is not visible or does not exist.
        Throws:
        java.io.IOException - on failure to read the ref or check it for visibility.

      • service

        private void service​(PacketLineOut pckOut)
              throws java.io.IOException
        Throws:
        java.io.IOException

      • lsRefsV2

        private void lsRefsV2​(PacketLineOut pckOut)
               throws java.io.IOException
        Throws:
        java.io.IOException

      • wantedRefs

        private java.util.Map<java.lang.String,​ObjectId> wantedRefs​(FetchV2Request req)
                                                           throws java.io.IOException
        Throws:
        java.io.IOException

      • fetchV2

        private void fetchV2​(PacketLineOut pckOut)
              throws java.io.IOException
        Throws:
        java.io.IOException

      • serveOneCommandV2

        private boolean serveOneCommandV2​(PacketLineOut pckOut)
                           throws java.io.IOException
        Throws:
        java.io.IOException

      • getV2CapabilityAdvertisement

        private java.util.List<java.lang.String> getV2CapabilityAdvertisement()

      • serviceV2

        private void serviceV2​(PacketLineOut pckOut)
                throws java.io.IOException
        Throws:
        java.io.IOException

      • refIdSet

        private static java.util.Set<ObjectId> refIdSet​(java.util.Collection<Ref> refs)

      • sendAdvertisedRefs

        public void sendAdvertisedRefs​(RefAdvertiser adv)
                        throws java.io.IOException,
                               ServiceMayNotContinueException
        Generate an advertisement of available refs and capabilities.
        Parameters:
        adv - the advertisement formatter.
        Throws:
        java.io.IOException - the formatter failed to write an advertisement.
        ServiceMayNotContinueException - the hook denied advertisement.

      • sendAdvertisedRefs

        public void sendAdvertisedRefs​(RefAdvertiser adv,
                               @Nullable
                               java.lang.String serviceName)
                        throws java.io.IOException,
                               ServiceMayNotContinueException
        Generate an advertisement of available refs and capabilities.
        Parameters:
        adv - the advertisement formatter.
        serviceName - if not null, also output "# service=serviceName" followed by a flush packet before the advertisement. This is required in v0 of the HTTP protocol, described in Git's Documentation/technical/http-protocol.txt.
        Throws:
        java.io.IOException - the formatter failed to write an advertisement.
        ServiceMayNotContinueException - the hook denied advertisement.
        Since:
        5.0

      • sendMessage

        public void sendMessage​(java.lang.String what)
        Send a message to the client, if it supports receiving them.

        If the client doesn't support receiving messages, the message will be discarded, with no other indication to the caller or to the client.

        Parameters:
        what - string describing the problem identified by the hook. The string must not end with an LF, and must not contain an LF.
        Since:
        3.1

      • getMessageOutputStream

        public java.io.OutputStream getMessageOutputStream()
        Get an underlying stream for sending messages to the client
        Returns:
        an underlying stream for sending messages to the client, or null.
        Since:
        3.1

      • getDepth

        public int getDepth()
        Returns the clone/fetch depth. Valid only after calling recvWants(). A depth of 1 means return only the wants.
        Returns:
        the depth requested by the client, or 0 if unbounded.
        Since:
        4.0

      • getFilterBlobLimit

        @Deprecated
public final long getFilterBlobLimit()
        Deprecated.
        Use getFilterSpec() instead
        Deprecated synonym for getFilterSpec().getBlobLimit().
        Returns:
        filter blob limit requested by the client, or -1 if no limit
        Since:
        5.3

      • getFilterSpec

        public final FilterSpec getFilterSpec()
        Returns the filter spec for the current request. Valid only after calling recvWants(). This may be a no-op filter spec, but it won't be null.
        Returns:
        filter requested by the client
        Since:
        5.4

      • getPeerUserAgent

        public java.lang.String getPeerUserAgent()
        Get the user agent of the client.

        If the client is new enough to use agent= capability that value will be returned. Older HTTP clients may also supply their version using the HTTP User-Agent header. The capability overrides the HTTP header if both are available.

        When an HTTP request has been received this method returns the HTTP User-Agent header value until capabilities have been parsed.

        Returns:
        user agent supplied by the client. Available only if the client is new enough to advertise its user agent.
        Since:
        4.0

      • parseWants

        private void parseWants​(PackStatistics.Accumulator accumulator)
                 throws java.io.IOException
        Throws:
        java.io.IOException

      • want

        private void want​(RevObject obj)

      • checkNotAdvertisedWants

        private static void checkNotAdvertisedWants​(UploadPack up,
                                            java.util.List<ObjectId> notAdvertisedWants,
                                            java.util.Collection<Ref> visibleRefs)
                                     throws java.io.IOException
        Throws:
        java.io.IOException

      • importantRefsFirst

        static java.util.stream.Stream<Ref> importantRefsFirst​(java.util.Collection<Ref> visibleRefs)

      • refToObjectId

        private static ObjectId refToObjectId​(Ref ref)

      • objectIdToRevCommit

        @Nullable
private static RevCommit objectIdToRevCommit​(RevWalk walk,
                                             ObjectId objectId)
        Translate an object id to a RevCommit.
        Parameters:
        walk - walk on the relevant object storae
        objectId - Object Id
        Returns:
        RevCommit instance or null if the object is missing

      • objectIdToRevObject

        @Nullable
private static RevObject objectIdToRevObject​(RevWalk walk,
                                             ObjectId objectId)
        Translate an object id to a RevObject.
        Parameters:
        walk - walk on the relevant object storage
        objectId - Object Id
        Returns:
        RevObject instance or null if the object is missing

      • addCommonBase

        private void addCommonBase​(RevObject o)

      • wantSatisfied

        private boolean wantSatisfied​(RevObject want)
                       throws java.io.IOException
        Throws:
        java.io.IOException

      • sendPack

        private void sendPack​(PackStatistics.Accumulator accumulator,
                      FetchRequest req,
                      @Nullable
                      java.util.Collection<Ref> allTags,
                      java.util.List<ObjectId> unshallowCommits,
                      java.util.List<ObjectId> deepenNots,
                      PacketLineOut pckOut)
               throws java.io.IOException
        Send the requested objects to the client.
        Parameters:
        accumulator - where to write statistics about the content of the pack.
        req - request in process
        allTags - refs to search for annotated tags to include in the pack if the #OPTION_INCLUDE_TAG capability was requested.
        unshallowCommits - shallow commits on the client that are now becoming unshallow
        deepenNots - objects that the client specified using --shallow-exclude
        pckOut - output writer
        Throws:
        java.io.IOException - if an error occurred while generating or writing the pack.

      • sendPack

        private void sendPack​(ProgressMonitor pm,
                      PacketLineOut pckOut,
                      java.io.OutputStream packOut,
                      FetchRequest req,
                      PackStatistics.Accumulator accumulator,
                      @Nullable
                      java.util.Collection<Ref> allTags,
                      java.util.List<ObjectId> unshallowCommits,
                      java.util.List<ObjectId> deepenNots)
               throws java.io.IOException
        Send the requested objects to the client.
        Parameters:
        pm - progress monitor
        pckOut - PacketLineOut that shares the output with packOut
        packOut - packfile output
        req - request being processed
        accumulator - where to write statistics about the content of the pack.
        allTags - refs to search for annotated tags to include in the pack if the #OPTION_INCLUDE_TAG capability was requested.
        unshallowCommits - shallow commits on the client that are now becoming unshallow
        deepenNots - objects that the client specified using --shallow-exclude
        Throws:
        java.io.IOException - if an error occurred while generating or writing the pack.

      • findSymrefs

        private static void findSymrefs​(RefAdvertiser adv,
                                java.util.Map<java.lang.String,​Ref> refs)

      • addTagChain

        private void addTagChain​(RevObject o,
                         PackWriter pw)
                  throws java.io.IOException
        Throws:
        java.io.IOException