\subsection{Recall across document categories: others, news and social}

The recall for Wikipedia entities in Table \ref{tab:name} ranged from 61.8\% (canonicals) to 77.9\% (name-variants).  Table \ref{tab:source-delta} shows how recall is distributed across document categories. For Wikipedia entities, across all entity profiles, others have a higher recall followed by news, and then by social.  While the recall for news  ranged from 76.4\% to 98.4\%, the recall for social documents ranged from 65.7\% to 86.8\%. In Twitter entities, however, the pattern is different. In canonicals (and their partials), social documents achieve higher recall than news. 

%This indicates that social documents refer to Twitter entities by their canonical names (user names) more than news do. In name- variant partial, news achieve better results than social. The difference in recall between canonicals and name-variants show that news do not refer to Twitter entities by their user names, they refer to them by their display names.

Overall, across all entities types and all entity profiles, others achieve higher recall than news, and  news, in turn, achieve higher recall than social documents. 

% This suggests that social documents are the hardest  to retrieve.  This  makes sense since social posts such as tweets and blogs are short and are more likely to point to other resources, or use short informal names.

%%NOTE TABLE REMOVED:\\\\

%

%We computed four percentage increases in recall (deltas)  between the

%different entity profiles (Table \ref{tab:source-delta2}). The first

%delta is the recall percentage between canonical partial  and

%canonical. The second  is  between name= variant and canonical. The

%third is the difference between name-variant partial  and canonical

%partial and the fourth between name-variant partial and

%name-variant. we believe these four deltas offer a clear meaning. The

%delta between name-variant and canonical measn the percentage of

%documents that the new name variants retrieve, but the canonical name

%does not. Similarly, the delta between  name-variant partial and

%partial canonical-partial means the percentage of document-entity

%pairs that can be gained by the partial names of the name variants. 

% The  biggest delta  observed is in Twitter entities between partials

% of all name variants and partials of canonicals (93\%). delta. Both

% of them are for news category.  For Wikipedia entities, the highest

% delta observed is 19.5\% in cano\_part - cano followed by 17.5\% in

% all\_part in relevant. 

  \subsection{Entity Types: Wikipedia and Twitter}

Table \ref{tab:name} shows the difference between Wikipedia and Twitter entities.  Wikipedia entities' canonical achieves a recall of 70\%, and canonical partial  achieves a recall of 86.1\%. This is an increase in recall of 16.1\%. By contrast, the increase in recall of name-variant partial over name-variant is 8.3.   

%The high increase in recall when moving from canonical names  to their partial names, in comparison to the lower increase when moving from all name variants to their partial names can be explained by saturation. This is to mean that documents have already been extracted by the different name variants and thus using their partial names does not bring in many new relevant documents. 

One interesting observation is that, For Wikipedia entities, canonical partial achieves better recall than name-variant in both cleansed and raw corpus.  %In the raw extraction, the difference is about 3.7. 

In Twitter entities, however, it is different. Both canonical and their partials perform the same and the recall is very low. Canonical  and canonical partial are the same for Twitter entities because they are one word strings. For example in https://twitter.com/roryscovel, ``roryscovel`` is the canonical name and its partial is also the same.  

%The low recall is because the canonical names of Twitter entities are not really names; they are usually arbitrarily created user names. It shows that  documents  refer to them by their display names, rarely by their user name, which is reflected in the name-variant recall (67.9\%). The use of name-variant partial increases the recall to 88.2\%.

The tables in \ref{tab:name} and \ref{tab:source-delta} show recall for Wikipedia entities are higher than for Twitter. Generally, at both aggregate and document category levels, we observe that recall increases as we move from canonicals to canonical partial, to name-variant, and to name-variant partial. The only case where this does not hold is in the transition from Wikipedia's canonical partial to name-variant. At the aggregate level(as can be inferred from Table \ref{tab:name}), the difference in performance between  canonical  and name-variant partial is 31.9\% on all entities, 20.7\% on Wikipedia entities, and 79.5\% on Twitter entities. This is a significant performance difference. 

%% TODO: PERHAPS SUMMARY OF DISCUSSION HERE

\section{Impact on classification}

  In here, we present results showing how  the choices in corpus, entity types, and entity profiles impact these latest stages of the pipeline.  In tables \ref{tab:class-vital} and \ref{tab:class-vital-relevant}, we show the performances in max-F. 

\begin{table*}

\caption{vital performance under different name variants(upper part from cleansed, lower part from raw)}

\begin{center}

\begin{tabular}{ll@{\quad}lllllll}

\hline

%&\multicolumn{1}{l}{\rule{0pt}{12pt}}&\multicolumn{1}{l}{\rule{0pt}{12pt}cano}&\multicolumn{1}{l}{\rule{0pt}{12pt}canonical partial }&\multicolumn{1}{l}{\rule{0pt}{12pt}name-variant }&\multicolumn{1}{l}{\rule{0pt}{50pt}name-variant partial}\\[5pt]

  &&cano&cano-part&all  &all-part \\

   all-entities &max-F& 0.241&0.261&0.259&0.265\\

%	      &SU&0.259  &0.258 &0.263 &0.262 \\	

   Wikipedia &max-F&0.252&0.274& 0.265&0.271\\

%	      &SU& 0.261& 0.259&  0.265&0.264 \\

   twitter &max-F&0.105&0.105&0.218&0.228\\

%     &SU &0.105&0.250& 0.254&0.253\\

\hline

\hline

  all-entities &max-F & 0.240 &0.272 &0.250&0.251\\

%	  &SU& 0.258   &0.151  &0.264  &0.258\\

   Wikipedia&max-F &0.257&0.257&0.257&0.255\\

%   &SU	     & 0.265&0.265 &0.266 & 0.259\\

   twitter&max-F &0.188&0.188&0.208&0.231\\

%	&SU&    0.269 &0.250 &0.250&0.253\\

\hline

\end{tabular}

\end{center}

\label{tab:class-vital}

\end{table*}

  \begin{table*}

\caption{vital-relevant performances under different name variants(upper part from cleansed, lower part from raw)}

\begin{center}

\begin{tabular}{ll@{\quad}lllllll}

\hline

%&\multicolumn{1}{l}{\rule{0pt}{12pt}}&\multicolumn{1}{l}{\rule{0pt}{12pt}canonical}&\multicolumn{1}{l}{\rule{0pt}{12pt}canonical partial }&\multicolumn{1}{l}{\rule{0pt}{12pt}name-variant }&\multicolumn{1}{l}{\rule{0pt}{50pt}name-variant partial}\\[5pt]

 &&cano&cano-part&all  &all-part \\

   all-entities &max-F& 0.497&0.560&0.579&0.607\\

%	      &SU&0.468  &0.484 &0.483 &0.492 \\	

   Wikipedia &max-F&0.546&0.618&0.599&0.617\\