��Yf{��@s�ddddgZddlZddlZddlZddlZddlmZddlmZdd l m Z dd l mZddl mZddl m Z dd l mZddl mZddl mZdd�Zejeje�dd�dddfD�Zedek rJdd�ZxeD]Zejee�q0WGdd�de�Zfidd�Zdd�ZGdd�de�Zd d!�Zd"d#�ZGd$d%�d%e�ZGd&d'�d'e�Z d(e j!e j"fd)e j#e j$fiZ%Gd*d�de�Z&Gd+d,�d,e'�Z(Gd-d�de�Z)d.d/�Z*id0d1�Z+dddd2d3d4�Z,Gd5d6�d6e�Z-Gd7d8�d8e�Z.d2d9d:�Z/Gd;d<�d<e)�Z0Gd=d>�d>e)�Z1Gd?d@�d@e1�Z2GdAdB�dBe)�Z3GdCdD�dDe)�Z4GdEdF�dFe)�Z5GdGdH�dHe)�Z6e+dIdJdKdLdMdNdOdPdQdRdSdTdUdVdWdXdYdZd[d\f�Z7Gd]d^�d^e7�Z8e+d_dKdLdMdNdRd`dadbdcdddXdddedfdf�Z9e+dgdNdMdRf�Z:e+dhdidjdkdldmdndodpdqdrdsf�Z;djdtdpdtdrdtdldudmduie;_<Gdvdh�dhe;�Z=Gdwd�de&�Z>e>jdxej?�e>jdyej?�e>jdzej@e3�e>jd{ejAe1�e>jd|ejBe1�e>jd}ejCe1�e>jd~ejDe1�e>jdejEe2�e>jd�ejFe4�e>jd�ejGe=�e>jd�ee8�e>jd�eHe9�e>jd8e.e6�e>jd:e/e:�e>jd6e-e5�e>jdud�e0d�d��e>jdtd�d��dS)��BaseManager�SyncManager� BaseProxy�Token�N)�time)� format_exc�)� connection)�context)�pool)�process)� reduction)�util)�get_contextcCstj|j|j�ffS)N)�array�typecode�tobytes)�a�r�=/opt/alt/python35/lib64/python3.5/multiprocessing/managers.py�reduce_array%srcCs(g|]}tti|��qSr)�type�getattr)�.0�namerrr� )s r�items�keys�valuescCstt|�ffS)N)�list)�objrrr�rebuild_as_list+sr!c@sLeZdZdZdZdd�Zdd�Zd d �Zdd�Zd S)rz4 Type to uniquely indentify a shared object �typeid�address�idcCs!||||_|_|_dS)N)r"r#r$)�selfr"r#r$rrr�__init__:szToken.__init__cCs|j|j|jfS)N)r"r#r$)r%rrr�__getstate__=szToken.__getstate__cCs|\|_|_|_dS)N)r"r#r$)r%�staterrr�__setstate__@szToken.__setstate__cCs#d|jj|j|j|jfS)Nz %s(typeid=%r, address=%r, id=%r))� __class__�__name__r"r#r$)r%rrr�__repr__CszToken.__repr__N)r"r#r$) r+� __module__�__qualname__�__doc__� __slots__r&r'r)r,rrrrr4scCsN|j||||f�|j�\}}|dkr;|St||��dS)zL Send a message to manager using connection `c` and return response z#RETURNN)�send�recv�convert_to_error)�cr$� methodname�args�kwds�kind�resultrrr�dispatchKs r:cCsN|dkr|S|dkr&t|�S|dkr@td|�Std�SdS)Nz#ERRORz #TRACEBACKz#UNSERIALIZABLEzUnserializable message: %s zUnrecognized message type)�RemoteError� ValueError)r8r9rrrr3Us r3c@seZdZdd�ZdS)r;cCs)ddddt|jd�ddS)N� �-�Kr)�strr6)r%rrr�__str__bszRemoteError.__str__N)r+r-r.rArrrrr;asr;cCsIg}x<t|�D].}t||�}t|�r|j|�qW|S)z4 Return a list of names of methods of `obj` )�dirr�callable�append)r �tempr�funcrrr�all_methodsisrGcCsdd�t|�D�S)zP Return a list of names of methods of `obj` which do not start with '_' cSs&g|]}|ddkr|�qS)r�_r)rrrrrrxs z"public_methods..)rG)r rrr�public_methodstsrIc @seZdZdZdddddddd d g Zdd�Zd d�Zdd�Zdd�Zdd�Z dd�Z dd�Zdd�Zdedede iZ dd�Zd d!�Zd"d#�Zd$d%�Zd&d'�Zd(d)�Zd*d+�Zd,d-�Zd.d/�Zd0S)1�ServerzM Server class which runs in a process controlled by a manager object �shutdown�create�accept_connection�get_methods� debug_info�number_of_objects�dummy�incref�decrefcCs�||_tj|�|_t|\}}|d|dd�|_|jj|_ddffi|_i|_t j �|_dS)Nr#Zbacklog��0)�registryr�AuthenticationString�authkey�listener_client�listenerr#� id_to_obj�id_to_refcount� threading�RLock�mutex)r%rVr#rX� serializer�Listener�Clientrrrr&�s zServer.__init__cCs�tj�|_|tj�_zqtjd|j�}d|_|j �y*x#|jj �sq|jjd�qOWWntt fk r�YnXWdtjtjkr�tjd�tjt_tjt_tjd�XdS)z( Run the server forever �targetTrNzresetting stdout, stderrr)r]�Event� stop_eventr�current_process�_manager_server�Thread�accepter�daemon�start�is_set�wait�KeyboardInterrupt� SystemExit�sys�stdout� __stdout__r�debug� __stderr__�stderr�exit)r%rirrr� serve_forever�s zServer.serve_foreverc Csgx`y|jj�}Wntk r-wYnXtjd|jd|f�}d|_|j�qWdS)Nrcr6T)rZZaccept�OSErrorr]rh�handle_requestrjrk)r%r4�trrrri�s zServer.accepterc'Csud}}}yWtj||j�tj||j�|j�}|\}}}}t||�}Wn!tk r�dt�f} YnGXy||||�}Wn!tk r�dt�f} Yn Xd|f} y|j| �Wn�tk rf} zdy|jdt�f�Wntk r#YnXt j d| �t j d|�t j d| �WYdd} ~ XnX|j�dS)z) Handle a new connection Nz #TRACEBACKz#RETURNzFailure to send message: %rz ... request was %rz ... exception was %r)r Zdeliver_challengerXZanswer_challenger2r� Exceptionrr1r�info�close)r%r4�funcnamer9�request�ignorer6r7rF�msg�errrry�s2 #zServer.handle_requestc*Cs%tjdtj�j�|j}|j}|j}x�|jj �s y,d}}|�}|\}}} } ||\}}}||kr�t d|t|�|f��t||�} y| | | �}Wn1t k r}zd|f}WYdd}~XnpX|o|j|d�}|re|j|||�\}}t||j|�}d||ff}nd|f}Wn�t k r|dkr�dt�f}n\y8|j|}|||||| | �}d|f}Wn!t k r�dt�f}YnXYnXtk r7tjdtj�j�tjd �Yn!t k rWdt�f}YnXyMy||�Wn8t k r�}z|d t�f�WYdd}~XnXWq7t k r}zUtjdtj�j�tjd|�tjd |�|j�tjd�WYdd}~Xq7Xq7WdS)zQ Handle requests from the proxies in a particular process/thread z$starting server thread to service %rNz+method %r of %r object is not in exposed=%rz#ERRORz#PROXYz#RETURNz #TRACEBACKz$got EOF -- exiting thread serving %rrz#UNSERIALIZABLEzexception in thread serving %rz ... message was %rz ... exception was %rr)rrsr]�current_threadrr2r1r[rerl�AttributeErrorrrr{�getrLrr#r�fallback_mapping�EOFErrorrprvr|r})r%�connr2r1r[r5r r�identr6r7�exposedZ gettypeidZfunction�resr�r�r"ZridentZrexposed�tokenZ fallback_funcr9rrr�serve_client�sl * zServer.serve_clientcCs|S)Nr)r%r�r�r rrr�fallback_getvalueszServer.fallback_getvaluecCs t|�S)N)r@)r%r�r�r rrr�fallback_str"szServer.fallback_strcCs t|�S)N)�repr)r%r�r�r rrr� fallback_repr%szServer.fallback_reprrAr,z #GETVALUEcCsdS)Nr)r%r4rrrrQ.szServer.dummycCs�|j��g}t|jj��}|j�xV|D]N}|dkr6|jd||j|t|j|d�dd�f�q6Wdj|�SWdQRXdS)zO Return some info --- useful to spot problems with refcounting rUz %s: refcount=%s %srNr?r=) r_rr[r�sortrDr\r@�join)r%r4r9rr�rrrrO1s *zServer.debug_infocCst|j�dS)z* Number of shared objects r)�lenr[)r%r4rrrrP@szServer.number_of_objectscCsXzCytjd�|jd�Wnddl}|j�YnXWd|jj�XdS)z' Shutdown this process z!manager received shutdown message�#RETURNNr)r�N)rrsr1� traceback� print_excre�set)r%r4r�rrrrKFs zServer.shutdowncOs|j��|j|\}}}}|dkr<|d} n|||�} |dkrct| �}|dk r�t|�t|�}dt| �} tjd|| �| t|�|f|j| <| |j kr�d|j | <|j || �| t|�fSWdQRXdS)z> Create a new shared object and return its id Nrz%xz&%r callable returned object with id %r)r_rVrIrr$rrsr�r[r\rR�tuple)r%r4r"r6r7rCr��method_to_typeid� proxytyper r�rrrrLSs z Server.createcCst|j|jd�S)zL Return the methods of the shared object indicated by token r)r�r[r$)r%r4r�rrrrNvszServer.get_methodscCs-|tj�_|jd�|j|�dS)z= Spawn a new thread to serve this connection �#RETURNN)r�N)r]r�rr1r�)r%r4rrrrrM|s zServer.accept_connectioncCs&|j�|j|d7 Connect manager object to the server process rXNrQ) rYr�r�r�r:r�r�r�r�)r%rarbr�rrr�connect�szBaseManager.connectcCsO|dk r%t|�r%td��tjdd�\}}|jjdt|�jd|j|j |j |j|||f�|_dj dd �|jjD��}t|�jd ||j_|jj�|j�|j�|_ |j�tj|j_tj|t|�jd|j|j |j |j|jfdd�|_dS) z@ Spawn a server process for this manager object Nzinitializer must be a callableZduplexFrcr6�:css|]}t|�VqdS)N)r@)r�irrr� �sz$BaseManager.start..r>�exitpriorityr)rC� TypeErrorr ZPiper�ZProcessr�_run_serverr�r�r�r��_processr�Z _identityr+rrkr}r2r�r�r�r�r�Finalize�_finalize_managerr�rK)r%�initializer�initargs�reader�writerr�rrrrk�s&" zBaseManager.startc Csi|dk r||�|j||||�}|j|j�|j�tjd|j�|j�dS)z@ Create a server, report its address and run it Nzmanager serving at %r)�_Serverr1r#r}rr|rw) �clsrVr#rXr`r�r�r��serverrrrr��s zBaseManager._run_servercOsk|j|jd|j�}z)t|dd|f||�\}}Wd|j�Xt||j|�|fS)zP Create a new shared object; return the token and exposed tuple rXNrL)r�r�r�r:r}r)r%r"r6r7r�r$r�rrr�_creates )zBaseManager._createcCs;|jdk r7|jj|�|jj�s7d|_dS)zC Join the manager process (if it has been spawned) N)r�r��is_alive)r%�timeoutrrrr�szBaseManager.joinc CsA|j|jd|j�}zt|dd�SWd|j�XdS)zS Return some info about the servers shared objects and connections rXNrO)r�r�r�r:r})r%r�rrr�_debug_infoszBaseManager._debug_infoc CsA|j|jd|j�}zt|dd�SWd|j�XdS)z5 Return the number of shared objects rXNrP)r�r�r�r:r})r%r�rrr�_number_of_objects"szBaseManager._number_of_objectscCs#|jjtjkr|j�|S)N)r�r�r�r�rk)r%rrr� __enter__,s zBaseManager.__enter__cCs|j�dS)N)rK)r%�exc_type�exc_val�exc_tbrrr�__exit__2szBaseManager.__exit__cCs|j�r�tjd�y8||d|�}zt|dd�Wd|j�XWntk reYnX|jdd�|j�r�tjd�t|d�r�tjd �|j�|jdd �|j�r�tjd�t j |_ytj |=Wntk rYnXdS)zQ Shutdown the manager process; will be registered as a finalizer z#sending shutdown message to managerrXNrKr�g�?zmanager still alive� terminatez'trying to `terminate()` manager processg��?z#manager still alive after terminate)r�rr|r:r}r{r��hasattrr�r�r�r�r�_address_to_local�KeyError)rr#rXr(r�r�rrrr�5s. zBaseManager._finalize_managercCs|jS)N)r�)r%rrr�UszBaseManager.Tc s�d|jkr!|jj�|_�dkr3t�|pHt�dd�}|p`t�dd�}|r�x t|j��D]\}}q|W|||�f|j�<|r��fdd�} �| _t|�| �dS)z9 Register a typeid with the manager type r�N� _exposed_�_method_to_typeid_c s�tjd��|j�||�\}}�||jd|d|jd|�}|j|jd|j�}t|dd|jf�|S)Nz)requesting creation of a shared %r object�managerrXr�rS) rrsr�r�r�r�r#r:r$)r%r6r7r�Zexp�proxyr�)r�r"rrrErsz"BaseManager.register..temp) �__dict__r��copy� AutoProxyrrrr+�setattr) r�r"rCr�r�r�� create_method�keyr�rEr)r�r"r�registerWs zBaseManager.register)r+r-r.r/r�rJr�r&r�r�rk�classmethodr�r�r�r�r�r�r��staticmethodr��propertyr#r�rrrrr�s* $ c@s(eZdZdd�Zdd�ZdS)�ProcessLocalSetcCstj|dd��dS)NcSs |j�S)N)�clear)r rrrr��sz*ProcessLocalSet.__init__..)r�register_after_fork)r%rrrr&�szProcessLocalSet.__init__cCst|�ffS)N)r)r%rrr� __reduce__�szProcessLocalSet.__reduce__N)r+r-r.r&r�rrrrr��sr�c@s�eZdZdZiZej�Zdddddd�Zdd�Z fidd �Z d d�Zdd �Ze dd��Zdd�Zdd�Zdd�Zdd�Zdd�ZdS)rz. A base for proxies of shared objects NTc Cs*tj�Ntjj|jd�}|dkrStj�t�f}|tj|j)rr+r�r"r$)r%rrrr,%szBaseProxy.__repr__cCsBy|jd�SWn*tk r=t|�dd�dSYnXdS)zV Return representation of the referent (or a fall-back if that fails) r,Nrz; '__str__()' failed>r�)r�r{r�)r%rrrrA)s zBaseProxy.__str__)r+r-r.r/r�rZForkAwareThreadLockr�r&r�r�r�r�r�r�r�r�r�r,rArrrrr�s" cCs�ttj�dd�}|rB|j|jkrB|j|jdS|jdd�ojttj�dd�}|||d||�SdS)z� Function used for unpickling proxy objects. If possible the shared object is returned, or otherwise a proxy for it. rgNrrRTZ_inheritingF)rrrfr#r[r$�pop)rFr�r`r7r�rRrrrr�6sr�c Cs�t|�}y|||fSWntk r2YnXi}x%|D]}td||f|�q@Wt|tf|�}||_||||f<|S)zB Return a proxy type whose methods are given by `exposed` zLdef %s(self, *args, **kwds): return self._callmethod(%r, args, kwds))r�r��execrrr�)rr��_cacheZdic�meth� ProxyTyperrr� MakeProxyTypeKs r�Tc Cs�t|d}|dkrY||jd|�}zt|dd|f�}Wd|j�X|dkrz|dk rz|j}|dkr�tj�j}td|j |�}|||d|d|d|�} d| _ | S) z* Return an auto-proxy for `token` rNrXrNz AutoProxy[%s]r�rRT)rYr#r:r}r�rrfrXr�r"r�) r�r`r�rXr�rRr�r�r�r�rrrr�as r�c@s(eZdZdd�Zdd�ZdS)� NamespacecKs|jj|�dS)N)r��update)r%r7rrrr&szNamespace.__init__cCst|jj��}g}x:|D]2\}}|jd�s"|jd||f�q"W|j�d|jjdj|�fS)NrHz%s=%rz%s(%s)z, ) rr�r� startswithrDr�r*r+r�)r%rrErr�rrrr,�s zNamespace.__repr__N)r+r-r.r&r,rrrrr�~sr�c@sReZdZddd�Zdd�Zdd�Zdd �Zeee�Zd S)�ValueTcCs||_||_dS)N)� _typecode�_value)r%rr��lockrrrr&�s zValue.__init__cCs|jS)N)r�)r%rrrr��sz Value.getcCs ||_dS)N)r�)r%r�rrrr��sz Value.setcCs dt|�j|j|jfS)Nz %s(%r, %r))rr+r�r�)r%rrrr,�szValue.__repr__N) r+r-r.r&r�r�r,r�r�rrrrr��s r�cCstj||�S)N)r)rZsequencer�rrr�Array�src@sReZdZdZdd�Zdd�Zd d �Zdd�Zd d�ZdS)� IteratorProxy�__next__r1�throwr}cCs|S)Nr)r%rrr�__iter__�szIteratorProxy.__iter__cGs|jd|�S)Nr)r�)r%r6rrrr�szIteratorProxy.__next__cGs|jd|�S)Nr1)r�)r%r6rrrr1�szIteratorProxy.sendcGs|jd|�S)Nr)r�)r%r6rrrr�szIteratorProxy.throwcGs|jd|�S)Nr})r�)r%r6rrrr}�szIteratorProxy.closeN)rr1rr}) r+r-r.r�rrr1rr}rrrrr�src@sLeZdZd Zdddd�Zdd�Zd d �Zdd�ZdS)� AcquirerProxy�acquire�releaseTNcCs1|dkr|fn ||f}|jd|�S)Nr)r�)r%Zblockingr�r6rrrr�s!zAcquirerProxy.acquirecCs |jd�S)Nr)r�)r%rrrr�szAcquirerProxy.releasecCs |jd�S)Nr)r�)r%rrrr��szAcquirerProxy.__enter__cCs |jd�S)Nr)r�)r%r�r�r�rrrr��szAcquirerProxy.__exit__)rr)r+r-r.r�rrr�r�rrrrr�s rc@sLeZdZdZddd�Zd d �Zdd�Zdd d�ZdS)�ConditionProxyrrrm�notify� notify_allNcCs|jd|f�S)Nrm)r�)r%r�rrrrm�szConditionProxy.waitcCs |jd�S)Nr )r�)r%rrrr �szConditionProxy.notifycCs |jd�S)Nr )r�)r%rrrr �szConditionProxy.notify_allcCs�|�}|r|S|dk r/t�|}nd}d}xF|s�|dk rj|t�}|dkrjP|j|�|�}q>W|S)Nr)�_timerm)r%Z predicater�r9ZendtimeZwaittimerrr�wait_for�s zConditionProxy.wait_for)rrrmr r )r+r-r.r�rmr r rrrrrr�s rc@sIeZdZdZdd�Zdd�Zd d �Zddd �ZdS)� EventProxyrlr�r�rmcCs |jd�S)Nrl)r�)r%rrrrl�szEventProxy.is_setcCs |jd�S)Nr�)r�)r%rrrr��szEventProxy.setcCs |jd�S)Nr�)r�)r%rrrr��szEventProxy.clearNcCs|jd|f�S)Nrm)r�)r%r�rrrrm�szEventProxy.wait)rlr�r�rm)r+r-r.r�rlr�r�rmrrrrr �s r c@sseZdZdZddd�Zdd �Zd d�Zedd ��Zedd��Z edd��Z dS)�BarrierProxy�__getattribute__rm�abort�resetNcCs|jd|f�S)Nrm)r�)r%r�rrrrm�szBarrierProxy.waitcCs |jd�S)Nr)r�)r%rrrr�szBarrierProxy.abortcCs |jd�S)Nr)r�)r%rrrr�szBarrierProxy.resetcCs|jdd�S)Nr�parties)r)r�)r%rrrr�szBarrierProxy.partiescCs|jdd�S)Nr� n_waiting)r)r�)r%rrrr�szBarrierProxy.n_waitingcCs|jdd�S)Nr�broken)r)r�)r%rrrr�szBarrierProxy.broken)rrmrr)r+r-r.r�rmrrr�rrrrrrrr�src@s:eZdZdZdd�Zdd�Zdd �Zd S)�NamespaceProxyr�__setattr__�__delattr__cCsB|ddkr tj||�Stj|d�}|d|f�S)NrrHr�r)�objectr)r%r�� callmethodrrr�__getattr__�szNamespaceProxy.__getattr__cCsH|ddkr#tj|||�Stj|d�}|d||f�S)NrrHr�r)rrr)r%r�r�rrrrr�szNamespaceProxy.__setattr__cCsB|ddkr tj||�Stj|d�}|d|f�S)NrrHr�r)rrr)r%r�rrrrr�szNamespaceProxy.__delattr__N)rrr)r+r-r.r�rrrrrrrr�src@s=eZdZdZdd�Zdd�Zeee�ZdS) � ValueProxyr�r�cCs |jd�S)Nr�)r�)r%rrrr�szValueProxy.getcCs|jd|f�S)Nr�)r�)r%r�rrrr� szValueProxy.setN)r�r�)r+r-r.r�r�r�r�r�rrrrrsr� BaseListProxy�__add__�__contains__�__delitem__�__getitem__�__len__�__mul__�__reversed__�__rmul__�__setitem__rD�count�extend�index�insertr��remove�reverser��__imul__c@s(eZdZdd�Zdd�ZdS)� ListProxycCs|jd|f�|S)Nr')r�)r%r�rrr�__iadd__szListProxy.__iadd__cCs|jd|f�|S)Nr,)r�)r%r�rrrr,szListProxy.__imul__N)r+r-r.r.r,rrrrr-sr-� DictProxyr�r�r�Zhas_key�popitem� setdefaultr�� ArrayProxy� PoolProxyZapplyZapply_asyncr}ZimapZimap_unorderedr��mapZ map_async�starmapZ starmap_asyncr�ZAsyncResult�Iteratorc@s(eZdZdd�Zdd�ZdS)r3cCs|S)Nr)r%rrrr�5szPoolProxy.__enter__cCs|j�dS)N)r�)r%r�r�r�rrrr�7szPoolProxy.__exit__N)r+r-r.r�r�rrrrr34sc@seZdZdZdS)ra( Subclass of `BaseManager` which supports a number of shared object types. The types registered are those intended for the synchronization of threads, plus `dict`, `list` and `Namespace`. The `multiprocessing.Manager()` function creates started instances of this class. N)r+r-r.r/rrrrr>s �QueueZ JoinableQueuerd�Lockr^� Semaphore�BoundedSemaphore� Condition�Barrier�Poolr�dictr�r�F)I�__all__rpr]rZqueuerrr�r�r r rrr rrrr�Z view_typesrr!Z view_typerrr:r3r{r;rGrIrJr�rarbZXmlListenerZ XmlClientrYrr�r�rr�r�r�r�r�rrrrr rrrrr-r/r2Z BasePoolProxyr�r3rr7rdr8r^r9r:r;r<r=r>rrrr�s� ��