MetaScroll Search Blog

ProgrammableWeb chose MetaScroll as their “Mashup of the Day” for August 30th 2010! This is the first coverage of MetaScroll on international websites.

Thank you very much, guys! Great to see that others love MetaScroll as much as I do!

Your mouse or trackpads are great ways to skim through your search results on MetaScroll! Anyhow, some folks prefer to not travel with their hands between keyboard and mouse/trackpad when searching. They’d ike to leave them on the keyboard.

For those keyboard afficiandos MetaScroll just got a lot better! You now aren’t dependend on your mouse anymore! Instead, you can move between search results using your cursor keys. To open a search result, just use “Enter” or the “Space”-bar.

Folling you’ll find a list of shortcuts:

With the searchbox unfocussed:

• Just start typing for a new search, the searchbox gets focussed automatically
• You can hit “Esc” to focus the searchbox, without overwriting the previous search term

With the searchbox focussed:

• “Enter” performs a search, selecting the first search result
• “Up”/”Down” switches to the next/previous search suggestion
• “Tab” unfocusses the searchbox and selects the first search result, without doing a search

Moving in search results:

• “Up” or “k” select the previous search result (YES! I like vi, too!)
• “Down” or “j” select the next one
• “Enter” or “Space” open the currently selected result

You may have to allow popups for the domain www.metascroll.com to be able to open a search result using the keyboard. These get opened in a new window or tab, depending on your browser setting.

I really hope you like these new features as much as we do! If you have any suggestions for other shortcuts, please, don’t hesitate to let me know!

I am proud to annonce that by now a new feature just got released! MetaScroll now hints various non-HTML MIME-types to you by displaying an according icon (taken from the great KDE Oxygen set).

You can try it yourself by doing a search for e.g. some PDFs. Hinted MIME-types include, but are not limited to:

• PDF
• MP3, WAV, AAC
• WMV, AVI, MOV
• DOC, XLS, CSV, RTF
• TTF, Torrent, Diff

If you have a MIME-type you’d like to see included, please drop me a line!

At first, I used the classic allrounder, the Apache2-webserver. But since its memory footprint isn’t the smallest, especially with all the modules I use (like mod_deflate, mod_jk, mod_proxy, mod_php, mod_rewrite, mod_ssl, etc), a service like MetaScroll generates quite a load on the machine, even with very few users. This is because of many requests being proxied over the service, so that only one search generates at least three requests on the server, not counting the suggestions.

The solution I chose was the use of the lightweight nginx-webserver. It isn’t as full-featured and modular as the Apache, but it comes with everything I need and a lot more performance while using only a percentage of Apache’s memory.

You can find the relevant settings I use below.

Nginx

• Gzip compression
  gzip on;
gzip_http_version 1.0;
gzip_vary on;
gzip_comp_level 6;
gzip_proxied any;
gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
gzip_buffers 16 8k;

• Turn off server tokens
  server_name_in_redirect off;
server_tokens off;

• Various TCP-settings
  sendfile on;
  tcp_nopush on;
  tcp_nodelay off;
  keepalive_timeout 10;

In the cause of this blog, I will try to give you an impression of the technology behind MetaScroll and the ropes that I’ve jumped through, to give you a top performance for your we search. Hopefully you can learn a thing or two, to help you with YOUR projects.

The topics that I’ll try to cover are …

• Caching
• Document size reduction
• Webserver configuration
• Backend performance
• Frontend performance

So stay tuned for the things coming up!

MetaScroll is the attempt to build a modern version of the classic meta-search engines. These engines, though quite popular in the late 90s, early 2000s, lost their market share to Google, like all the other of the popular search engines at that time.

Although Google delivers great results and a good searching experiences, there still are issues with their web search:

• Ease of use. At the end of a result page, one has to manually find and click the “next”-button to get to the next page of web results. This process interrupts the visual scanning of the results.
• Search performance. While Google’s search result pages are undoubtingly fast, having to find the “next”-button and click it to get to the next page is a repetitive and slow task.
• Direct and relevant answers. Google does an incredible job finding the best websites regarding your search. But you still have to open the page with the information you’re looking for. And the info on this page could be days, weeks, or even years old when you are searching for something thats happening right now!
• Privacy. This is the most important issue. Google does not only store every search you make and when you make it, it also stores which results you click on and if or how much later you come bck to their site. The collected information, together with the data from other Google services like Mail, Calendar, Reader, etc lead to a very distinct profile of you and your interests.

Of course you’re free to use other search engines like Microsoft’s Bing or Yahoo!, but the issues named above are valid those as well, to a greater or smaller extend.

So, how does MetaScroll fit into the picture? What can it offer regarding the issues named above?

• Ease of use. There is now “Next page”-button on MetaScroll you’d have to find and point your mouse at. The result just keep coming while you scroll down the page.
• Search performance. The process of scrolling down the page doesn’t get interrupted with new pages. MetaScroll delivers the results in one large list that grows while you scroll.
• Direct and relevant answers. Classic search engine results are integrated with relevant Wikipedia articles and other people’s messages from Twitter. This way you get a 360° view on the topic you’re searching for.
• Privacy. MetaScroll does all it can to ensure that no search engine has the possibility to identify you while you’re searching. And of course no personal information is stored by MetaScroll itself

.So you see, MetaScroll gives a great deal about improving your searching experience. And if there is anything you see that could fit MetaScroll and it’s philosophy, or that you would need to definitly do the switch from yourcurrent favorit search engine, you’re more than welcome to get in touch!

Let me know your thoughts!

Hendrik, Developer of MetaScroll.com