� ��f�mc@s�dZddlmZdZddlZddlZddlZddlZddlZddl Z ddl mZddlm Z mZmZddlmZdd lmZmZdd lmZddlmZedd �ZGdd�de�Zdd�Zdd�Zdd�Zdd�Z ej!ddfkrpddl"Z"e"j#Z$dd�Z%dd�Z&ne#Z$e Z%e Z&dd�Z'Gdd �d e�Z(Gd!d"�d"e)�Z*Gd#d$�d$e�Z+Gd%d&�d&e*�Z,dS('u�Refactoring framework. Used as a main program, this can refactor any number of files and/or recursively descend down directories. Imported as a module, this provides infrastructure to write your own refactoring tool. i(uwith_statementu#Guido van Rossum N(uchaini(udriverutokenizeutoken(u find_root(upytreeupygram(u btm_utils(ubtm_matchercCs�t|ggdg�}tjj|j�}g}xnttj|��D]W}|jd�rI|jd�rI|r�|dd�}n|j |dd��qIqIW|S(uEReturn a sorted list of all available fix names in the given package.u*ufix_u.pyiNii��( u __import__uosupathudirnameu__file__usortedulistdiru startswithuendswithuappend(u fixer_pkgu remove_prefixupkgu fixer_diru fix_namesuname((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyuget_all_fix_names!suget_all_fix_namescBs|EeZdZdS(u _EveryNodeN(u__name__u __module__u__qualname__(u __locals__((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyu _EveryNode.su _EveryNodecCs�t|tjtjf�rC|jdkr3t�nt|jg�St|tj�rt|j rkt |j �St�nt|tj�r�t�}x5|j D]*}x!|D]}|jt |��q�Wq�W|St d|��dS(uf Accepts a pytree Pattern Node and returns a set of the pattern types which will match first. u$Oh no! I don't understand pattern %sN(u isinstanceupytreeuNodePatternuLeafPatternutypeuNoneu _EveryNodeusetuNegatedPatternucontentu_get_head_typesuWildcardPatternuupdateu Exception(upaturupux((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyu_get_head_types2s u_get_head_typescCstjt�}g}x�|D]�}|jr�yt|j�}Wntk r_|j|�Yq�XxU|D]}||j|�qgWq|jdk r�||jj|�q|j|�qWx:t t jjj �t jj�D]}||j|�q�Wt|�S(u^ Accepts a list of fixers and returns a dictionary of head node type --> fixer list. N(ucollectionsudefaultdictulistupatternu_get_head_typesu _EveryNodeuappendu_accept_typeuNoneuchainupygramupython_grammaru symbol2numberuvaluesutokensuextendudict(u fixer_listu head_nodesueveryufixeruheadsu node_type((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyu_get_headnode_dictNs" u_get_headnode_dictcs �fdd�t�d�D�S(uN Return the fully qualified names for fixers in the package pkg_name. csg|]}�d|�qS(u.((u.0ufix_name(upkg_name(u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyu ks u+get_fixers_from_package..F(uget_all_fix_namesuFalse(upkg_name((upkg_nameu5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyuget_fixers_from_packagegsuget_fixers_from_packagecCs|S(N((uobj((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyu _identitynsu _identityicCs|jdd�S(Nu u (ureplace(uinput((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyu_from_system_newlinesusu_from_system_newlinescCs*tjdkr"|jdtj�S|SdS(Nu (uosulinesepureplace(uinput((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyu_to_system_newlineswsu_to_system_newlinesc s�d}tjtj|�j��fdd�}ttjtj tj f�}t�}ydx]|�\}}||kr�q`q`|tjkr�|r�Pnd }q`|tjkr�|dkr�|�\}}|tjks�|dkr�Pn|�\}}|tjks|dkrPn|�\}}|tjkrV|dkrV|�\}}nxa|tjkr�|j|�|�\}}|tjks�|dkr�Pn|�\}}qYWq`Pq`Wntk r�YnXt|�S( Ncst��}|d|dfS(Nii(unext(utok(ugen(u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyuadvance�su(_detect_future_features..advanceufromu __future__uimportu(u,FT(uFalseutokenizeugenerate_tokensuiouStringIOureadlineu frozensetutokenuNEWLINEuNLuCOMMENTusetuSTRINGuTrueuNAMEuOPuaddu StopIteration(usourceuhave_docstringuadvanceuignoreufeaturesutpuvalue((ugenu5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyu_detect_future_features�sD u_detect_future_featurescBs|EeZdZdZdS(u FixerErroruA fixer could not be loaded.N(u__name__u __module__u__qualname__u__doc__(u __locals__((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyu FixerError�su FixerErrorcBso|EeZdZid4d6d4d6ZdZdZd3d3dd�Zdd�Z d d �Z dd�Zd d�Zdd�Z d4d4dd�Zd4d4dd�Zdd�Zd4d4dd�Zdd�Zd4dd�Zdd�Zdd �Zd3d4d3d!d"�Zd3d#d$�Zd%Zd&Zd'd(�Zd)d*�Zd+d,�Zd-d.�Zd/d0�Zd1d2�Zd3S(5uRefactoringTooluprint_functionuwrite_unchanged_filesuFixufix_cCs�||_|pg|_|jj�|_|dk rI|jj|�n|jdretj|_ ntj |_ |jjd�|_g|_ tjd�|_g|_d|_tj|j dtjd|j�|_|j�\|_|_g|_tj�|_g|_g|_ x}t!|j|j�D]f}|j"rT|jj#|�q2||jkrv|jj$|�q2||jkr2|j j$|�q2q2Wt%|j�|_&t%|j �|_'dS(u�Initializer. Args: fixer_names: a list of fixers to import options: an dict with configuration. explicit: a list of fixers to run even if they are explicit. uprint_functionuwrite_unchanged_filesuRefactoringTooluconvertuloggerNF((ufixersuexplicitu_default_optionsucopyuoptionsuNoneuupdateupygramu!python_grammar_no_print_statementugrammarupython_grammarugetuwrite_unchanged_filesuerrorsuloggingu getLoggeruloggeru fixer_loguFalseuwroteudriveruDriverupytreeuconvertu get_fixersu pre_orderu post_orderufilesubmu BottomMatcheruBMu bmi_pre_orderubmi_post_orderuchainu BM_compatibleu add_fixeruappendu_get_headnode_dictubmi_pre_order_headsubmi_post_order_heads(uselfufixer_namesuoptionsuexplicitufixer((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyu__init__�s< uRefactoringTool.__init__cCs�g}g}x�|jD]�}t|iidg�}|jdd�d}|j|j�rx|t|j�d�}n|jd�}|jdjdd�|D��}yt ||�}Wn(t k r�td ||f��YnX||j|j �} | jr:|jdk r:||jkr:|jd |�qn|jd|�| jdkri|j| �q| jd kr�|j| �qtd| j��qWtjd�} |jd| �|jd| �||fS(uInspects the options to load the requested patterns and handlers. Returns: (pre_order, post_order), where pre_order is the list of fixers that want a pre-order AST traversal, and post_order is the list that want post-order traversal. u*u.iNu_ucSsg|]}|j��qS((utitle(u.0up((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyu �s u.RefactoringTool.get_fixers..uCan't find %s.%suSkipping implicit fixer: %suAdding transformation: %supreupostuIllegal fixer order: %ru run_orderukeyi��T(ufixersu __import__ursplitu startswithuFILE_PREFIXulenusplituCLASS_PREFIXujoinugetattruAttributeErroru FixerErroruoptionsu fixer_loguexplicituTrueulog_messageu log_debuguorderuappenduoperatoru attrgetterusort(uselfupre_order_fixersupost_order_fixersufix_mod_pathumodufix_nameupartsu class_nameu fix_classufixerukey_func((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyu get_fixers�s8# uRefactoringTool.get_fixerscOs�dS(uCalled when an error occurs.N((uselfumsguargsukwds((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyu log_errorsuRefactoringTool.log_errorcGs'|r||}n|jj|�dS(uHook to log a message.N(uloggeruinfo(uselfumsguargs((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyulog_messages uRefactoringTool.log_messagecGs'|r||}n|jj|�dS(N(uloggerudebug(uselfumsguargs((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyu log_debugs uRefactoringTool.log_debugcCsdS(uTCalled with the old version, new version, and filename of a refactored file.N((uselfuold_textunew_textufilenameuequal((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyuprint_output!suRefactoringTool.print_outputcCsPxI|D]A}tjj|�r5|j|||�q|j|||�qWdS(u)Refactor a list of files and directories.N(uosupathuisdirurefactor_diru refactor_file(uselfuitemsuwriteu doctests_onlyudir_or_file((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyurefactor&s uRefactoringTool.refactorc Cs�tjd}x�tj|�D]�\}}}|jd|�|j�|j�xe|D]]}|jd�rWtjj|�d|krWtjj||�} |j | ||�qWqWWdd�|D�|dd�As u0RefactoringTool.refactor_dir..N( uosuextsepuwalku log_debugusortu startswithupathusplitextujoinu refactor_file( uselfudir_nameuwriteu doctests_onlyupy_extudirpathudirnamesu filenamesunameufullname((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyurefactor_dir/s uRefactoringTool.refactor_dircCs�yt|d�}Wn<tk rQ}z|jd||�dSWYdd}~XnXztj|j�d}Wd|j�Xt|dd|��}t |j ��|fSWdQXdS(uG Do our best to decode a Python source file correctly. urbuCan't open %s: %sNiuruencoding(NN(uopenuIOErroru log_erroruNoneutokenizeudetect_encodingureadlineucloseu_open_with_encodingu_from_system_newlinesuread(uselfufilenameufuerruencoding((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyu_read_python_sourceCsu#RefactoringTool._read_python_sourcec Cs|j|�\}}|dkr%dS|d7}|r�|jd|�|j||�}|jsl||kr�|j|||||�q|jd|�ni|j||�}|js�|r�|jr�|jt|�dd �|d|d|�n|jd|�dS( uRefactors a file.Nu uRefactoring doctests in %suNo doctest changes in %siuwriteuencodinguNo changes in %si��( u_read_python_sourceuNoneu log_debugurefactor_docstringuwrite_unchanged_filesuprocessed_fileurefactor_stringuwas_changedustr(uselfufilenameuwriteu doctests_onlyuinputuencodinguoutpututree((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyu refactor_fileSs uRefactoringTool.refactor_filecCs�t|�}d|kr*tj|j_nzby|jj|�}WnEtk r�}z%|jd||jj |�dSWYdd}~XnXWd|j|j_X||_ |jd|�|j||�|S(uFRefactor a given input string. Args: data: a string holding the code to be refactored. name: a human-readable name for use in error/log messages. Returns: An AST corresponding to the refactored input stream; None if there were errors during the parse. uprint_functionuCan't parse %s: %s: %sNuRefactoring %s( u_detect_future_featuresupygramu!python_grammar_no_print_statementudriverugrammaruparse_stringu Exceptionu log_erroru __class__u__name__ufuture_featuresu log_debugu refactor_tree(uselfudataunameufeaturesutreeuerr((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyurefactor_stringjs uRefactoringTool.refactor_stringcCs�tjj�}|ro|jd�|j|d�}|jsI||kr_|j|d|�q�|jd�nS|j|d�}|js�|r�|jr�|jt |�d|�n |jd�dS(NuRefactoring doctests in stdinuuNo doctest changes in stdinuNo changes in stdin( usysustdinureadu log_debugurefactor_docstringuwrite_unchanged_filesuprocessed_fileurefactor_stringuwas_changedustr(uselfu doctests_onlyuinputuoutpututree((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyurefactor_stdin�s uRefactoringTool.refactor_stdinc Cs�x-t|j|j�D]}|j||�qW|j|j|j��|j|j|j��|jj|j ��}x�t |j��rdx�|jjD]�}||kr�||r�||j dtjjdd�|jr�||j dtjj�nx\t||�D]G}|||kr9||j|�nyt|�Wntk r^wYnX|jr}||jkr}qn|j|�}|r|j||�}|dk rV|j|�x9|j�D]+}|js�g|_n|jj|�q�W|jj|j ��}x?|D]4} | |kr7g|| >> u... c Cs�g}d}d}d}d}x4|jdd�D] }|d7}|j�j|j�r�|dk r�|j|j||||��n|}|g}|j|j�} |d| �}q1|dk r |j||j �s�|||j j �dkr |j|�q1|dk r8|j|j||||��nd}d}|j|�q1W|dk r�|j|j||||��ndj|�S(u�Refactors a docstring, looking for doctests. This returns a modified version of the input string. It looks for doctests, which start with a ">>>" prompt, and may be continued with "..." prompts, as long as the "..." is indented the same as the ">>>". (Unfortunately we can't use the doctest module's parser, since, like most parsers, it is not geared towards preserving the original source.) iukeependsiNu uT( uNoneu splitlinesuTrueulstripu startswithuPS1uextendurefactor_doctestufinduPS2urstripuappendujoin( uselfuinputufilenameuresultublockublock_linenouindentulinenoulineui((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyurefactor_docstring(s: u"RefactoringTool.refactor_docstringc s�y�j||��}Wn�tk r�}zj�jjtj�rpx*|D]}�jd|jd��qJWn�jd|||j j |�|SWYdd}~XnX�j||�r�t|�j dd �}|d|d�||dd�} }| dg|dkst| ��|djd�s@|dd7>>" (possibly indented), while the remaining lines start with "..." (identically indented). u Source: %su u+Can't parse docstring in %s line %s: %s: %sNukeependsiics!g|]}��j|�qS((uPS2(u.0uline(uindentuself(u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyu ms u4RefactoringTool.refactor_doctest..Ti��i��(uparse_blocku ExceptionuloggeruisEnabledForulogginguDEBUGu log_debugurstripu log_erroru __class__u__name__u refactor_treeustru splitlinesuTrueuAssertionErroruendswithuPS1upop( uselfublockulinenouindentufilenameutreeuerrulineunewuclipped((uindentuselfu5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyurefactor_doctestSs& )#&u RefactoringTool.refactor_doctestcCs|jrd}nd}|js4|jd|�n1|jd|�x|jD]}|j|�qNW|jr�|jd�x!|jD]}|j|�q�Wn|jrt|j�dkr�|jd�n|jdt|j��x0|jD]"\}}}|j|||�q�WndS( Nuwereu need to beuNo files %s modified.uFiles that %s modified:u$Warnings/messages while refactoring:iuThere was 1 error:uThere were %d errors:(uwroteufilesulog_messageu fixer_loguerrorsulen(uselfuwereufileumessageumsguargsukwds((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyu summarizeps$ uRefactoringTool.summarizecCs1|jj|j|||��}t�|_|S(u�Parses a block into a tree. This is necessary to get correct line number / offset information in the parser diagnostics and embedded into the parse tree. (udriveruparse_tokensu wrap_toksu frozensetufuture_features(uselfublockulinenouindentutree((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyuparse_block�s!uRefactoringTool.parse_blockccs�tj|j||�j�}xe|D]]\}}\}}\} } }||d7}| |d7} ||||f| | f|fVq%WdS(u;Wraps a tokenize stream to systematically modify start/end.iN(utokenizeugenerate_tokensu gen_linesu__next__(uselfublockulinenouindentutokensutypeuvalueuline0ucol0uline1ucol1u line_text((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyu wrap_toks�s (uRefactoringTool.wrap_toksccs�||j}||j}|}xo|D]g}|j|�rT|t|�d�Vn4||j�dkrrdVntd||f��|}q'WxdVq�dS(u�Generates lines as expected by tokenize from a list of lines. This strips the first len(indent + self.PS1) characters off each line. Nu uline=%r, prefix=%ru(uPS1uPS2u startswithulenurstripuAssertionError(uselfublockuindentuprefix1uprefix2uprefixuline((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyu gen_lines�s uRefactoringTool.gen_linesNF( u__name__u __module__u__qualname__uFalseu_default_optionsuCLASS_PREFIXuFILE_PREFIXuNoneu__init__u get_fixersu log_errorulog_messageu log_debuguprint_outputurefactorurefactor_diru_read_python_sourceu refactor_fileurefactor_stringurefactor_stdinu refactor_treeutraverse_byuprocessed_fileu write_fileuPS1uPS2urefactor_docstringurefactor_doctestu summarizeuparse_blocku wrap_toksu gen_lines(u __locals__((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyuRefactoringTool�s: 4( O+ uRefactoringToolcBs|EeZdZdS(uMultiprocessingUnsupportedN(u__name__u __module__u__qualname__(u __locals__((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyuMultiprocessingUnsupported�suMultiprocessingUnsupportedcse|EeZdZ�fdd�Zd d d�fdd�Z�fdd�Z�fdd �Z�S(uMultiprocessRefactoringToolcs/tt|�j||�d|_d|_dS(N(usuperuMultiprocessRefactoringToolu__init__uNoneuqueueuoutput_lock(uselfuargsukwargs(u __class__(u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyu__init__�s u$MultiprocessRefactoringTool.__init__ics[|dkr(tt��j|||�Syddl�Wntk rRt�YnX�jdk rqtd��n�j ��_�j ��_��fdd�t|�D�}z;x|D]}|j �q�Wtt��j|||�Wd�jj�x$t|�D]}�jjd�q Wx'|D]}|j�r*|j�q*q*Wd�_XdS(Niiu already doing multiple processescs%g|]}�jd�j��qS(utarget(uProcessu_child(u.0ui(umultiprocessinguself(u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyu �s u8MultiprocessRefactoringTool.refactor..(usuperuMultiprocessRefactoringToolurefactorumultiprocessinguImportErroruMultiprocessingUnsupporteduqueueuNoneuRuntimeErroru JoinableQueueuLockuoutput_lockurangeustartujoinuputuis_alive(uselfuitemsuwriteu doctests_onlyu num_processesu processesupui(u __class__(umultiprocessinguselfu5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyurefactor�s2 u$MultiprocessRefactoringTool.refactorc so|jj�}xY|dk rj|\}}ztt|�j||�Wd|jj�X|jj�}qWdS(N(uqueueugetuNoneusuperuMultiprocessRefactoringToolu refactor_fileu task_done(uselfutaskuargsukwargs(u __class__(u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyu_child�su"MultiprocessRefactoringTool._childcsE|jdk r(|jj||f�ntt|�j||�SdS(N(uqueueuNoneuputusuperuMultiprocessRefactoringToolu refactor_file(uselfuargsukwargs(u __class__(u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyu refactor_file�su)MultiprocessRefactoringTool.refactor_fileF(u__name__u __module__u__qualname__u__init__uFalseurefactoru_childu refactor_file(u __locals__((u __class__u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyuMultiprocessRefactoringTool�s uMultiprocessRefactoringTool(-u__doc__u __future__uwith_statementu __author__uosusysulogginguoperatorucollectionsuiou itertoolsuchainupgen2udriverutokenizeutokenu fixer_utilu find_rootuupytreeupygramu btm_utilsubuubtm_matcherubmuTrueuget_all_fix_namesu Exceptionu _EveryNodeu_get_head_typesu_get_headnode_dictuget_fixers_from_packageu _identityuversion_infoucodecsuopenu_open_with_encodingu_from_system_newlinesu_to_system_newlinesu_detect_future_featuresu FixerErroruobjectuRefactoringTooluMultiprocessingUnsupporteduMultiprocessRefactoringTool(((u5/opt/alt/python33/lib64/python3.3/lib2to3/refactor.pyu sF (��