usage notes on STATIC_ROOT, STATIC_URL, MEDIA_ROOT, MEDIA_URL in Django v1.3+

In case I get confused again when I come back later.

• Difference of *_ROOT and *_URL

1.   *_ROOT are paths in the file system that actually stores the files, which are absolute paths.
2. *_URL are urls in the HTTP requests. Meant to be used in templates.

• Difference of STATIC_* and MEDIA_*

1. STATIC_*  are for .js, .css and background images(.jpg, .png, .gif) that construct the webpages.
2. MEDIA_* are media files (picture, music etc) that are uploaded by users or the like.

This separation is more logic than technical, and only introduced inv1.3 see release notes. Pictures can be served either in STATIC_ROOT  for  MEDIA_ROOT  as long as they are referenced with the corresponding *_URL. In a word, STATIC_*  are for infrastructure files, while MEDIA_ROOT are for content files(e.g photos in a photo gallery site)

• Typical settings

1. MEDIA_ROOT = os.path.join(PROJECT_ROOT_PATH, ‘media/’)
2. MEDIA_URL = ‘/media/’
3. STATIC_ROOT = os.path.join(PROJECT_ROOT_PATH, ‘static/’)
4. STATIC_URL = ‘/static/’

• Static-specific

1. If ‘django.contrib.staticfiles.finders.FileSystemFinder’ is enabled, STATICFILES_DIRS are searched by the development server.
2. If ‘django.contrib.staticfiles.finders. AppDirectoriesFinderis is enabled, static/ subfolder for each app (i.e. app/static) are searched by the development server.
3. Run ‘manage.py collectstatic’ would copy static file from the above folders to STATIC_ROOT. This is for 3rd servers like nginx, Apache etc. The development server does not use it.

• Issues between the development server and 3rd servers (nginx, apache etc.)

1. *_URL is for both the development server and 3rd servers (apache etc).
2. *_ROOT is a way to collect things together for 3rd servers, and 3rd servers would only look into *_ROOT for files when configured in their own server config files. The development server does not care about them.
3. By default, the development server look into the subfolders (static/) under each app for static files, but never care about media files cause it does not server media files, but template files only(js, css are needed for templates thus being searched).

The following code would enable the development server to server media files.from django.conf import settings

urlpatterns += patterns(”,
(r’^media/(?P<path>.*)$’, ‘django.views.static.serve’, {‘document_root’: settings.MEDIA_ROOT}),
)

2D Water Effect in WTL

I just published a piece of code onto http://www.codeproject.com.

=-=-=-=-=
Powered by Blogilo

c++ class for drawing oscilloscope waveforms

Recently I invested some time in oscilloscope waveforms drawing. The algorithm is simple:

– a target shape(petagon, clock starting with low/high level) is calculated with a given surrounding rectangle.

For ease of use,  further wrapping is provided for drawing a sequence of shapes.

The code is tested with VC++ 10.

Firecleaner: a python3 script to remove duplicate Firefox bookmark entries

If you are like me, using multiple internet browsers such as IE8/9, FireFox3/4, Google Chrome on both Windows XP/7 and Ubuntu, You are going to find Xmarks  indispensible.The only pain is that bookmarks may be duplicated many times.

I finally could not bear it and come up with this solution with my favorite  script language python:

1, export bookmarks from Firefox

2, run the below script on the exported file and get an out.html file

3. delete all entries in the Firefox bookmark menu(NOT bookmark toolbar)

4. import the out.html file

Running it on my bookmarks, I get:

"Of 5142 entries scanned, 4014 are duplicated and removed"

Nice.

 

# -*- coding: utf-8 -*-

class fireclean:
    
    """
        this script clean up duplicated firefox bookmarks entries
    """
    
    def __init__(self, fin = "bookmarks.html", fout = "out.html", flog = "leaveout.html"):
    
        """
            fin:  path to the original bookmark file exported from firefox
            fout: path to the generated file
            flog: path to the file containing removed lines
        """

        self.fin  = open(fin, encoding = 'utf-8')
        self.fout = open(fout, mode = 'w' , encoding = 'utf-8') 
        self.flog = open(flog, mode = 'w' , encoding = 'utf-8') 
        
        self.href_list   = []
        self.header_list = []
        
        self.total_lines   = 0    # line of <DT>s
        self.removed_lines = 0    # line of removed <DT>s
    
    def remove_u200b(self, s):
        
        """
            for some reason, U+200B is found in lines, probably inserted by XMarks
            it needs to be removed before checking duplicates
        """
        
        return s.replace('\u200b', '')
        

    def check_line(self, line, test, container):

        """
            parameters:
                line: the raw line to be checked
                test: substring of line to be tested
                container: container of tested strings

            algorithm:
                if test is found in container,
                then line is removed and written to flog
                otherwise to fout
        """
        
        is_duplicate = False
         
        print(test)
        
        if  not test in container:
            container.append(test)
            self.fout.write(line)
        else:
            self.flog.write(line)      
            is_duplicate = True
        
        return is_duplicate

    def clean(self, need_log = False):
    
        """
            parameters:
                need_log: print the number of scanned and removed entries if True

            algorithm:
                check duplicate HREFs for entries shown as items in FireFox bookmark manager
                check duplicate element names for entries shown as folders in FireFox bookmark manager
        """

        for line in self.fin:
            self.total_lines += 1
            line_ = self.remove_u200b(line).strip()

            if(line_.startswith('<DT><A')): # for bookmark items
                
                self.removed_lines += self.check_line(line, line_.split(' ', 2)[1], self.href_list)
                    
            elif(line_.startswith('<DT><H3')):   # for bookmark folders
                
                self.removed_lines += self.check_line(line, line_.rsplit('>', 2)[-2], self.header_list)
                    
            else:
                self.total_lines -= 1
                self.fout.write(line)
                
        self.fout.close()
        self.flog.close()
        
        if need_log:
            print("Of {0} entries scanned, {1} are duplicated and removed".format(self.total_lines, self.removed_lines ))
    
import sys
if __name__ == "__main__":
    if(len(sys.argv) > 1):
        fireclean(sys.argv[1]).clean(True)
    else:
        fireclean().clean(True)        

=-=-=-=-=

Powered by Blogilo

Cartesian Product with Boost.Preprocessor

Today is another exciting day as I finished the Preprocessor macro version of the Cartesian Product issue, unlike the template version, whose incarnation is a template class. This version is a switch-case structure.  Now that the infrastructure is ready, I’ll be able to concentrate on the concurrent studies again.

A great relief to several weeks of braining. and just in time for my holiday starting from tomorrow.

Code:

#ifndef CARTISIAN_PRODUCT_SWITCH_CASE_H

#define CARTISIAN_PRODUCT_SWITCH_CASE_H

#include <boost/preprocessor/seq/for_each_product.hpp>

#include <boost/preprocessor/seq/to_tuple.hpp>

/*

===========

Description:

===========

this macro construct a switch-case structrue by apply the Cartisian Product of

3-sequence to a clause that takes them as inputs.

i.e.

switch(NAME_OF_SWITCH_INDEX)

{

case ( COMPOSE_INDEX_FROM_SEQ): CLAUSE_FOR_EACH_CASE; break;

… …

case ( COMPOSE_INDEX_FROM_SEQ ): CLAUSE_FOR_EACH_CASE; break;

default: break;

}

for example:

Given 3 Sequences S1,S2,S3, if size(S1) =3,  size(S1) =2,  size(S1) =3,  the following something like

the following would be yielded:

switch(idx)

{

case ( ((0 & 0xF)<<8)| ((0 & 0xF)<<4)| ((0 & 0xF)) ): details::scanT<(0, 0, 0)>(); break;

case ( ((0 & 0xF)<<8)| ((0 & 0xF)<<4)| ((1 & 0xF)) ): details::scanT<(0, 0, 1)>(); break;

case ( ((0 & 0xF)<<8)| ((0 & 0xF)<<4)| ((2 & 0xF)) ): details::scanT<(0, 0, 2)>(); break;

case ( ((0 & 0xF)<<8)| ((1 & 0xF)<<4)| ((0 & 0xF)) ): details::scanT<(0, 1, 0)>(); break;

case ( ((0 & 0xF)<<8)| ((1 & 0xF)<<4)| ((1 & 0xF)) ): details::scanT<(0, 1, 1)>(); break;

case ( ((0 & 0xF)<<8)| ((1 & 0xF)<<4)| ((2 & 0xF)) ): details::scanT<(0, 1, 2)>(); break;

default: break;

}

===========

How to Use:

===========

// 1/2 change the following defines and call GENERATE_SWITCH_STRUCTURE_FOR_CARTISIAN_PRODUCTS

#define SIZE_OF_SEQ1 2

#define SIZE_OF_SEQ2 5

#define SIZE_OF_SEQ3 5

#define NAME_OF_SWITCH_INDEX g_method

#define CLAUSE_FOR_EACH_CASE(ONE_BOOST_PP_SEQ_TYPE_INSTANCE) \

details::scanT<ONE_BOOST_PP_SEQ_TYPE_INSTANCE>()

#define CLAUSE_FOR_DEFAULT_LABLE   std::wcout << L”this combination is not supported.”

// 2/2 run the macro

GENERATE_SWITCH_STRUCTURE_FOR_CARTISIAN_PRODUCTS

// ========       change this define if you prefer another algotithm         ========

// define variable name for switch

#ifndef NAME_OF_SWITCH_INDEX

#define NAME_OF_SWITCH_INDEX idx

#endif

//  define clause for case: label

#ifndef CLAUSE_FOR_EACH_CASE

#define CLAUSE_FOR_EACH_CASE(ONE_BOOST_PP_SEQ_TYPE_INSTANCE) ;

#endif

//  define clause for default: label

#ifndef CLAUSE_FOR_DEFAULT_LABLE

#define CLAUSE_FOR_DEFAULT_LABLE

#endif

*/

// Compose a 12-bit integer from 3-nibbles, which is represented by a BOOST_PP_TUPLE

//  i.e.    (X, Y, Z)    ==>     ( ((X & 0xF)<<8)| ((Y & 0xF)<<4)| ((Z & 0xF)) )

#define EXTRACT_FOUR_BIT_NIBBLE(N)  (N & 0xF)

#ifndef COMPOSE_INDEX_FROM_SEQ

#define COMPOSE_INDEX_FROM_SEQ(S) (\

(EXTRACT_FOUR_BIT_NIBBLE(BOOST_PP_SEQ_ELEM(0, S))<<8)|\

(EXTRACT_FOUR_BIT_NIBBLE(BOOST_PP_SEQ_ELEM(1, S))<<4)|\

(EXTRACT_FOUR_BIT_NIBBLE(BOOST_PP_SEQ_ELEM(2, S)))\

)

#endif

// ===== DO NOT CHANGE ANYTHING BELOW THIS LINE, UNLESS YOU KNOW WHAT YOU ARE DOING =====

// define all 3 parts of a switch-case block

#define BEGIN_SWITCH(METHOD_INDEX) switch(METHOD_INDEX) {

#define CASE(S) \

case COMPOSE_INDEX_FROM_SEQ(S):     \

CLAUSE_FOR_EACH_CASE(BOOST_PP_SEQ_ENUM(S)); break;

#define END_SWITCH \

default:    \

CLAUSE_FOR_DEFAULT_LABLE; }

// define 3 integer sequence  represented by a BOOST_PP_SEQUENCE

// i.e. N ==> (0) (1) (2)… (N-1) (N)

#define DECL_SELF(z, n, text) (n)

#define S1 BOOST_PP_REPEAT(SIZE_OF_SEQ1, DECL_SELF, UNUSED)

#define S2 BOOST_PP_REPEAT(SIZE_OF_SEQ2, DECL_SELF, UNUSED)

#define S3 BOOST_PP_REPEAT(SIZE_OF_SEQ3, DECL_SELF, UNUSED)

// apply macro CASE to each Cartisian product of (S1)(S2)(S3)

#define APPLY_ONE(r, product) CASE(product)

// run the macros    and  release all tokens used here

// example oupput:

/*

switch(idx)

{

case ( ((0 & 0xF)<<8)| ((0 & 0xF)<<4)| ((0 & 0xF)) ): details::scanT<(0, 0, 0)>(); break;

case ( ((0 & 0xF)<<8)| ((0 & 0xF)<<4)| ((1 & 0xF)) ): details::scanT<(0, 0, 1)>(); break;

case ( ((0 & 0xF)<<8)| ((0 & 0xF)<<4)| ((2 & 0xF)) ): details::scanT<(0, 0, 2)>(); break;

case ( ((0 & 0xF)<<8)| ((1 & 0xF)<<4)| ((0 & 0xF)) ): details::scanT<(0, 1, 0)>(); break;

case ( ((0 & 0xF)<<8)| ((1 & 0xF)<<4)| ((1 & 0xF)) ): details::scanT<(0, 1, 1)>(); break;

case ( ((0 & 0xF)<<8)| ((1 & 0xF)<<4)| ((2 & 0xF)) ): details::scanT<(0, 1, 2)>(); break;

default: break;

}

*/

#define GENERATE_SWITCH_STRUCTURE_FOR_CARTISIAN_PRODUCTS \

BEGIN_SWITCH(NAME_OF_SWITCH_INDEX) \

BOOST_PP_SEQ_FOR_EACH_PRODUCT(APPLY_ONE, (S1)(S2)(S3)) \

END_SWITCH

/*

#undef COMPOSE_INDEX

#undef BEGIN_SWITCH

#undef CASE

#undef END_SWITCH

#undef APPLY_ONE

#undef DECL_SELF

#undef S1

#undef S2

#undef S3

#undef SIZE_OF_SEQ1

#undef SIZE_OF_SEQ2

#undef SIZE_OF_SEQ3

#undef NAME_OF_SWITCH_INDEX

#undef CLAUSE_FOR_EACH_CASE

*/

#endif

A Day of Achievement – Cartesian Product with Boost.MPL

Today is a day worthy of celebration, as I finally solved a problem I’ve been working on for quite some time.  See it here.

Code:

#ifndef TRI_TYPE_CARTESIAN_PRODUCT_H

#define TRI_TYPE_CARTESIAN_PRODUCT_H


#include <boost/mpl/vector.hpp>


#include <typeinfo>


namespace recursive

{

    using boost::is_same;

    using boost::mpl::begin;

    using boost::mpl::end;

    using boost::mpl::next;        

    using boost::mpl::if_;  

    using boost::mpl::deref;    

    using boost::mpl::size;

    using boost::mpl::advance;  

    using boost::mpl::int_;  

    

    namespace detail

    {

        unsigned int total_recursions = 0;


        struct end_of_recursion_tag 

        {

            static void Run()

            {

                std::cout << "end of " 

                     << total_recursions 

                     << " recursions\n"

                    ;

                    

                total_recursions = 0;

            }

        };    

    }

    

    // generate a Cartesian Product of 3 type sequences and apply TestFunc on each generated type

    template <

        class UTypes,    // Forward Sequence, e.g. boost::mpl::vector

        class VTypes,    // Forward Sequence, e.g. boost::mpl::vector

        class WTypes,    // Forward Sequence, e.g. boost::mpl::vector

        class TestFunc  // class type that has a nested templated run() member function

    >

    struct CartesianProduct

    {

        // forward declaration

        template <

            class UIterator,

            class VIterator,

            class WIterator 

        >   

        class Generate;


        // this class implements recursion body 

        template <

            class UIterator,

            class VIterator,

            class WIterator

        >            

        struct Next

        {

            // u_begin is not necessary 😉

            // it would be cheaper not to pre-declare all of them since we force evaluation

            // however this dramatically increase the readability

            typedef typename begin<VTypes>::type v_begin;

            typedef typename begin<WTypes>::type w_begin;


            typedef typename end<UTypes>::type u_end;

            typedef typename end<VTypes>::type v_end;

            typedef typename end<WTypes>::type w_end;


            typedef typename next<UIterator>::type u_next;

            typedef typename next<VIterator>::type v_next;

            typedef typename next<WIterator>::type w_next;

        

            typedef typename if_< is_same<typename w_next, w_end>,

                                typename if_< is_same<v_next, v_end>,

                                    typename if_< is_same<u_next, u_end>,

                                        detail::end_of_recursion_tag,

                                        Generate< 

                                            u_next, 

                                            v_begin, 

                                            w_begin 

                                        >

                                    >::type,

                                    Generate< 

                                        UIterator, 

                                        v_next,

                                        w_begin 

                                    >

                                >::type,

                                Generate< 

                                    UIterator, 

                                    VIterator, 

                                    w_next

                                    >

                            >::type type;

        };

        

        //  this class run test on generated types in thos round and go to next*/

        template <

            class UIterator = typename begin<UTypes>::type,

            class VIterator = typename begin<VTypes>::type,

            class WIterator = typename begin<WTypes>::type

        >

        struct Generate

        {

            //  generate <<next>> target type         

            typedef typename Next<

                    UIterator, 

                    VIterator, 

                    WIterator 

                >::type next_type;


            static void Run()

            {

                // increment recursion counter                

                ++detail::total_recursions;

                

                // test on the generated types of this round of recursion

                TestFunc::run<

                     typename deref<UIterator>::type,

                     typename deref<VIterator>::type,

                     typename deref<WIterator>::type

                 >();


                // go to the next round of recursion

                next_type::Run();

            }

        };

    

        template <

            size_t UIndex = 0,

            size_t VIndex = 0,

            size_t WIndex = 0

        >

        struct GenerateByIndex

        {                

            static_assert(UIndex <= size<UTypes>::type::value, "too large stock container index");

            static_assert(VIndex <= size<VTypes>::type::value, "too large stock shrinker_group_number index");

            static_assert(WIndex <= size<WTypes>::type::value, "too large stock shrinker_group_size index");


            typedef typename begin<UTypes>::type u_begin;        

            typedef typename begin<VTypes>::type v_begin;

            typedef typename begin<WTypes>::type w_begin;         

    

            static void Run()

            {  

                Generate<

                    typename advance<u_begin, int_<UIndex> >::type,

                    typename advance<v_begin, int_<VIndex> >::type,

                    typename advance<w_begin, int_<WIndex> >::type                    

                >::Run();

            }

        };                

    };

    

    struct print_typeid

    {

        template<

            class U, 

            class V, 

            class W

        >

        static void run()

        {

            // print the typeinfo

            std::cout 

                <<  detail::total_recursions << ":"

                << typeid(U).name() << ","

                << typeid(V).name() << ","

                << typeid(W).name()

                << std::endl;  

        }            

    }; 

    

}//  namespace recursive_test


#endif

Zebra crosses Windows and Kubuntu 9.04

It’s comes so unexpected that a little tool of mine could run on Windows and Linux(Ubuntu 9.04) with no modification, only less pretty.

pointer to member functions

Though with a similiar grammar as that of free funstions and static class member functions,  pointers to member functions  are not addresses, but offsets – how can the addresses be determined when the class is not initialised on the stack/heap yet?

Below are two ways to  define and use member function pointers:

  1: // C-style version

  2: void shrink(raw_path_groups_type rpgs)

  3: {

  4:     // define the pointer

  5:     typedef detectiveT<walkerT, equal_predicateT> this_class;

  6:     typedef void (this_class::*func_type)(raw_path_groups_type); 

  7: 

  8:     boost::array<func_type, 5> f = 

  9:     {

 10:         &this_class::single_threaded_process, 

 11:         &this_class::concurrent_process_thread_group,

 12:         &this_class::concurrent_process_ppl,

 13:         &this_class::concurrent_process_tbb,

 14:         &this_class::concurrent_boost_task   

 15:     };  

 16:        

 17:     (this->*f[m_thread_mode])(rpgs); // invoke it

 18:     //alternatively, boost::bind(f[m_thread_mode], this, rpgs)();    

 19: }
alternatively, below is a boost::function version.
  1: // boost::function-style version

  2: void shrink(raw_path_groups_type rpgs)

  3: {

  4:     typedef detectiveT<walkerT, equal_predicateT> this_class;

  5:     typedef boost::function<void(this_class*, raw_path_groups_type)> funct_type;

  6:     

  7:     boost::array<funct_type, 5> f = 

  8:     {

  9:         // boost::mem_fn is necessary for VC++10 to resolve ambiguity with tr1;

 10:         // but necessary not for VC9 

 11:         boost::mem_fn(&this_class::single_threaded_process), 

 12:         boost::mem_fn(&this_class::concurrent_process_thread_group),

 13:         boost::mem_fn(&this_class::concurrent_process_ppl),

 14:         boost::mem_fn(&this_class::concurrent_process_tbb),

 15:         boost::mem_fn(&this_class::concurrent_boost_task)   

 16:     }; 

 17:     

 18:     f[m_thread_mode](this, rpgs); 

 19:     //alternatively, boost::bind(f[m_thread_mode], this, rpgs)();                

 20: } 

WinUSB 2.0 available

WinUSB 2.0 is released with WDK for Windows 7 and Windows Server 2008 R2. I have have time to check the difference it, at first glance of of winusb.h:

• macros like

IN/OUT 

are replaced with compiler directives 

__in/__out,

• __in_bcount(ValueLength)

is introduced to for bounds check.

 

Hope I could find time to check it more thoroughly and complete the WinUSB++ Series soon.

Hope.

C++ Template Metaprogramming

Having almost finished reading "C++ Template Metaprogramming: Concepts, Tools, and Techniques from Boost and Beyond" from cover to cover, I have to admit it’s not only a door that opens a new world before me,  but also a stage that lifts me higher, when I look back, to see things more clear.

Metaprogramming is about computing types to types, what are the points of doing that? Anyway, what we are doing in programming are virtually computing values to values(through functions), and types are actually the bounds that limit us. Without types(impossible to date) or with minimum types(as in dynamic languages like Python), life will seem to be much easier,  think about it, pick a symbol, say a, and use it as a file, a float, a string or whatever you like would be really fun. But it’s only a dream as yet.

Living in today’s world, being able to cross types, even only a few, is a forward step toward the Utopia of type-less world, and kind of liberation. That’s my understanding of the type computing part of metaprogramming.

Metaprogramming is also about integral values, which assists in type computing and crossing the compile-time and run-time world. However,  as we’ve been living long enough with values, this part is not so mind-drilling.