Ho Ho Ho!!!... Santa Claus is Coming To Town... on Mon, 2nd Nov 2009 11:15 am 

And those who are NOT GOOD boys… WON”T get a present from Santa For Christmas!!!… Ho Ho Ho!!!…

Bravo TR!!!… It looks liek WE HAVE become a Tour De Force TO BE reckon with!!! Ho Ho Ho!!!…

Sant Clause IS coming to Town this year TO GIVE US ALL… very Lovely Present for Christmas!!!… HO HO HO!!!

AlexTheGreat on Mon, 2nd Nov 2009 11:25 am 

This is getting dirty.
We demand an honest, clear and transparent response to this by those involved!
Everything happens for a reason and we deserve to know, be it for good or bad.
Lies will need more lies to cover up.
Don’t sink deeper….

XiSd Tay on Mon, 2nd Nov 2009 11:32 am 

Whoever your system admin is, kudos to him for a job well done!

I hope Darkness will not come around and cry foul and accuse TR of cooking this up.

Lets see what SPH has to say about this.

Commoner on Mon, 2nd Nov 2009 11:35 am 

Haha, quite obviously, SPH is grabbing all these articles to teach their journalists how to do objective reporting. They cannot put down their status to ask officially from TR, so no choice have to resort to this.

patzhou on Mon, 2nd Nov 2009 12:02 pm 

TR should see it as a compliment, that they had “motivated” ST’s attention and sneaky actions to crash TR’s site.

ST’s actions only reaffirm the status of TR as being the alternative media and voice for Singaporeans.

Keep up the great work, TR!

CPT on Mon, 2nd Nov 2009 12:15 pm 

I think they will take more action to bring down this site. who can question them?

Steve Wu on Mon, 2nd Nov 2009 12:17 pm 

The situation is getting quite interesting.

Even for website content meant to be publicly available, there is a convention to (voluntarily) prevent indiscriminate download of the entire or a substantial portion of a website. See, for example,
http://en.wikipedia.org/wiki/Robots_exclusion_standard

Does TR have a robots.txt? If so, there is a description of what constitutes authorized access. It does not matter if web crawlers do not observe it. The point is that it forms the basis of the complaint against the perpetrator using section 6 of the Computer Misuse Act (Cap 50A)

6. —(1) Subject to subsection (2), any person who knowingly —
(a) secures access without authority to any computer for the purpose of obtaining, directly or indirectly, any computer service;
(b) intercepts or causes to be intercepted without authority, directly or indirectly, any function of a computer by means of an electro-magnetic, acoustic, mechanical or other device; or
(c) uses or causes to be used, directly or indirectly, the computer or any other device for the purpose of committing an offence under paragraph (a) or (b), shall be guilty of an offence and shall be liable on conviction to a fine not exceeding $10,000 or to imprisonment for a term not exceeding 3 years or to both and, in the case of a second or subsequent
conviction, to a fine not exceeding $20,000 or to imprisonment for a term not exceeding 5 years or to both.

Freddy Song on Mon, 2nd Nov 2009 12:49 pm 

Should TR sue or launch an investigation?
Fortunately, most singaporeans may never ever know due to whatever reasons. Is this not great ? They no need to worry.

ILOVESINAGPORE on Mon, 2nd Nov 2009 12:49 pm 

Why dont you report to the police? Let the police do the investigation.

idunbelieveit on Mon, 2nd Nov 2009 12:50 pm 

SPH will never admit to anything.
Nobody will ever know.

Cheese on Mon, 2nd Nov 2009 1:00 pm 

SPH will never reply to TR’s request to clarify the matter.

TR, if you have the money, time and resources, why not just make a police report. If the police do not act on it then consider taking SPH to court on a private summon. The downside is you will not win the court case but you will make more people aware what kind of MSM SPH is.

By the way, that shorty MBT has not reply to your letter regarding the Town Council Act, right? Just like SPH, the PAP government will not give you a reply.

nocountryforoldmen on Mon, 2nd Nov 2009 1:02 pm 

Consider making a formal police report . It will be interesting see how the SPH reply to the police’s inquiry or how the police reply to the DDOS attack .

Boo on Mon, 2nd Nov 2009 1:08 pm 

SPH is effectively MIW’s public relations division…..so surprises here.

btan on Mon, 2nd Nov 2009 1:55 pm 

Why remind them? Why not just report to the police and let the police handle them?

wat? on Mon, 2nd Nov 2009 2:03 pm 

Are they really that stupid?

Teh on Mon, 2nd Nov 2009 2:03 pm 

No brainer, the attacker has to be some party linked to PAP. And they think they won’t be found out???

N on Mon, 2nd Nov 2009 2:04 pm 

Before the service interruption, go to news.google.com.sg and click on Singapore and you’ll find TR appearing as the top search is every single topic… recently, it’s down to one or two articles.

SPH is afraid, the are very afraid and I am afraid of what they will do next. It’s a big thorn in this government eye that anytime, anyone does a google, they see the truth reported on TR instead of the spins they put on MSM.

cy on Mon, 2nd Nov 2009 2:13 pm 

shame on you, SPH. is that how a “credible” media behaves?

admin on Mon, 2nd Nov 2009 2:30 pm 

Hi Cheese,

We never expect SPH to reply anyway. The main purpose of this article is just to expose them so that they will be extra careful the next time they decide to “grab” content from our site again.

As for making a police report, do you think the police will bother to take any action, somehow against SPH? For all you know, the report we made will be passed to SPH instead!

citizenofSG on Mon, 2nd Nov 2009 2:36 pm 

Latest show in Singapore. TR vs SPH . David vs Goliath.
Nah I think the matter will be dissipated in the wind.

admin on Mon, 2nd Nov 2009 2:47 pm 

Hi N,

You are spot on. That’s what we notice too. Before the DDOS attack, we have three articles out of the top 5 most read articles on Google News in Singapore. In fact, whenever there are more than 3 TR articles on Google News, they will always receive more hits than the entire msm combined.

Somebody actually wrote in to Google to complain that our site is not a “accreditated” news site and requested Google to take us off. Google refused and simply add a (blog) after our name.

These cowards will never dare to fight us on a level playing field because they know they will be demolished by us completely. That’s why they can only resort to such sneaky dirty tricks to bring us down and retard our growth.

Band of Brothers on Mon, 2nd Nov 2009 3:23 pm 

Now I see SPH in a different light, so much for “creditable & balance”.

Anonymous Coward on Mon, 2nd Nov 2009 4:36 pm 

@TR

Not approving of my comments? Why? It’s neither slanderous nor nitpicking. It’s just from a technical viewpoint.

Anonymous Coward.

Ryan on Mon, 2nd Nov 2009 4:38 pm 

hmm… y not make a police report since obviously they contravene the CMA?? A legit case against them unless they can prove someone else hack into their server, took control of their computer and did a DOS.

XiSd Tay on Mon, 2nd Nov 2009 4:42 pm 

A police report is basically oput of the question if the matter of the complaint is the unauthorised access because the archives are in PUBLIC DOMAIN and any Tom, Dick or Harry can access it.

However, if the grab is too much and causes the server to go down as a result, then technically the other section of the CMA might apply. However then again, the intention is one hard point to prove.

Even an idiot like me can think of why I had to grab TR’s site, not to mention the million dollar lawyers sitting at SPH doing nothing.

As long as the readers know what SPH have been doing, its better than wasting time and resources on making any report or sueing them.

However, admin can consider making a FORMAL COMPLAINT to the ISP (in this case starhub) and SPH. If they are a professional body as they claimed to be, then they are OBLIGATED to ‘investigate’ and reply.

Anonymous Coward on Mon, 2nd Nov 2009 4:46 pm 

@XiSD Tay,

No, your SP is not obligated to investigate as there are *absolutely* as far as what I can tell no proof here that there has been a denial of service and SP’s rarely spring into action unless you can prove it’s a spam-relay; participating in massive DDoS (part of a botnet, not the SPH concept of let’s view TR many times) and even so, you’d be hard press to get the SP to cut them off or anything and even in this case, the SP would most likely send an email to the network ops in the organization asking them to clean up.

Anonymous Coward

XiSd Tay on Mon, 2nd Nov 2009 4:56 pm 

@Anonymous Coward, the complaint is merely for record purposes and the idea is to let those involved reply.

I do agree that its a weak case but I merely suggested for the fun of it. Its not really a DDoS nor a SYNC attack, which would have been more damaging but I doubt they would be so stupid as to launch a SYNC or DDoS from a single IP. Any decent firewall will shut the attack off within seconds.

Honestly I think that now they have been exposed, TR’s stand and claims have been proven beyond reasonable doubt and readers already know what kind of organisation SPH is in reality.

admin on Mon, 2nd Nov 2009 4:57 pm 

Hi coward,

All your comments are approved. There is something wrong with the cache. Some comments are not visible immediately. We are still trying to fix the problem.

Anonymous Coward on Mon, 2nd Nov 2009 4:58 pm 

@XiSD Tay,

My previous post seems to have been deleted by TR for some odd reason. I have explained in my previous post why the article doesn’t hold water and why it isn’t technically sound. I’m all for pinning the blame on SPH but I do not see the need to make accusations which are not sound lest everyone here ends up looking like an idiot.

Anonymous Coward

Anonymous Coward on Mon, 2nd Nov 2009 5:01 pm 

@admin

Thanks for the info. o/

XiSd Tay on Mon, 2nd Nov 2009 5:41 pm 

Anonymous Coward, I am not accusing SPH of anything. The article itself stated that SPH was grabbing TR’s site and that is proven by the server’s log. Everyone can see that and I dont see anything technically unsound about stating a fact.

As to what they are doing or intending to do, your guess is as good as mine.

nameless on Mon, 2nd Nov 2009 6:01 pm 

DDOS attack stands for Distributed Denial of Service attack. Err…… if the “attack” is from one or two IP, where is the Distributed part?

CPT on Mon, 2nd Nov 2009 6:32 pm 

There is no point report to police. They are loyal servant of mm lky team.

KAM on Mon, 2nd Nov 2009 6:38 pm 

An official response:
It was an honest mistake.

But seriously, I don’t want to curse you, but sooner or later, they will come at you.
Stay cool!

XiSd Tay on Mon, 2nd Nov 2009 7:03 pm 

@nameless, nobody said it was a DDoS.

sheldon on Mon, 2nd Nov 2009 8:00 pm 

“nobody said it was a DDoS”

now i am confused. so was it a ddos or not? i thought the exchange between you and darkness was onto something then the whole chunk of it got deleted. why?

XiSd Tay on Mon, 2nd Nov 2009 8:43 pm 

Sheldon, this thread is NOT about DDoS.

sheldon on Mon, 2nd Nov 2009 8:58 pm 

okay. allow me to rephrase to understand you better.

there was indeed a ddos. but sph grabbing articles may or may not be ddos.

is it?

XiSd Tay on Mon, 2nd Nov 2009 9:03 pm 

@sheldon, let me rephrase my answer so you understand.

TR kenna DDoS has got NOTHING to do with SPH grabbing articles from TR’s server. The grabbing DID NOT created the DDoS.

They are 2 seperate incident although the timing was remarkably close

Michael on Mon, 2nd Nov 2009 9:17 pm 

Erection, oops, I mean Election is coming!

LKY once asked a student whether he would like to change his MP and today I am certain of that answer: I want to change my MP/s.

My constituency is Jurong GRC, anyone else?

sheldon on Mon, 2nd Nov 2009 9:27 pm 

okay. thanks for the clarification.

so from ip log and stuff, is it possible to say who’s responsible for the ddos?

MSM on Mon, 2nd Nov 2009 9:28 pm 

your other article talks about PAP ‘doing battle’ with opposition parties online. Guess PAP has started its attack–and not just with opp parties.

XiSd Tay on Mon, 2nd Nov 2009 10:05 pm 

@sheldon, it is very difficult to trace the person responsible just by server logs alone as far as DDoS or SYNC attack is concerned, maybe a reader here named Darkness can with his network of friends.

The server log will only show WHICH IPs have been attacking the server and that information alone is normally in hundreds or even thousands.

sheldon on Mon, 2nd Nov 2009 10:25 pm 

so, darkness, is it possible for you to find out who is responsible by looking at the server log?

XiSd Tay on Mon, 2nd Nov 2009 10:42 pm 

@sheldon, no need for Darkness to tell you who the person responsible was.

Its already stated clearly that their system admin has already identified the culprit to be from 203.116.231.234 which was traced back to Singapore Press Holdings.

A spider brought down TR? on Mon, 2nd Nov 2009 10:56 pm 

As an IT practitioner, I can’t help this. This article is making a gross technical misrepresentation. A spider does not make a DDOS. If a person uses a spider as a DDOS, then it must be the lamest and funniest DDOS ever. Unless the DDOS target is a 8086 on a telephone modem.

I hope TR isn’t a 8086 on a telephone modem. If not, when google, yahoo, msn, baidu, rednano or some other bot come along to poke, TR is going to be brought down again.

sheldon on Mon, 2nd Nov 2009 11:05 pm 

no i don’t mean the grabbing content affair, i meant the ddos.

paranoid TR on Mon, 2nd Nov 2009 11:14 pm 

Pardon, but I think TR is over-reacting to this grabber. If TR don’t handle this well, I’ll think twice to to surf your site in office. If one day, many of us surf at the same time from office and TR put up an article with our IP, then how?

I think TR is already scaring away everyone from SPH now. It’ll be the same case with other civil service and Temasek companies.

I wonder your anti-establishment has become anti-employee. What if the grabber from SPH is an ardent fan of the site? Really excuse me, but you’re being paranoid.

XiSd Tay on Mon, 2nd Nov 2009 11:24 pm 

@A spider brought down TR, you are not reading the article correctly. NOBODY claimed that the grabber or maybe a spider brought TR’s site down.

It merely stated that the incident was discovered and that evidence suggest someone in SPH was grabbing TR’s contents. NOWHERE in the article claim or suggest that it was this ‘grabbing’ that cause TR’s site to shut down.

@paranoid TR, can’t blame you for arrivng at that conclusion but if you have seen the FULL LOG, it would probably compel you to arrive at a different opinion.

From server log, the offending IP is reading EVERY SINGLE ARCHIVE on TR’s site CONSURRENTLY (ie: the conclusion that it was some web grabber software), would you open 100 browser in your office to view TR’s site?

paranoid TR on Mon, 2nd Nov 2009 11:28 pm 

I wonder that poor SPH guy must be in shit now. I suggest TR think properly before putting up his IP before all of us. You don’t even know if it could just be a reader using a grabber or many readers having night snack there.

Bruce W on Mon, 2nd Nov 2009 11:32 pm 

Making Police report? You must be out of your mind.

Long ago, opposition Tang LH made a police report on defamation against him from PAP members.
JBJ mentioned in rally that ‘He (Tang) told me that he’d made the police report’.
PAP sent the report to the press to publish, then claimed the publishing of the report defamed their integrity, and hence sued BOTH of them until bankrupt.

For detail, read http://www.tangtalk.com

XiSd Tay on Mon, 2nd Nov 2009 11:41 pm 

@paranoid TR, if someone or some group in SPH were indeed having night snack and a mass orgy, errr..ahem.. I meant mass surfing of TR, then they should have nothing to worry.

A spider brought down TR? on Mon, 2nd Nov 2009 11:58 pm 

Xisd, sorry but most of the article is about DDOS while the title is about grabbing. And “the anti-ddos firewall stop these attacks”. The attack means DDOS attack, not grabbing attack.

If it’s really DDOS, then the IPs will be spoofed and untrackable. There’ll be multiple sources to gather bandwidth for maximum firepower. There’s no way you can shut DDOS out unless you shut everyone out.

As a technical person, I guess it’s a spider from rednano. But even if it’s a grabber, so what? Road-warriors use it.

XiSd Tay on Tue, 3rd Nov 2009 12:12 am 

@A spider brought down TR, this article is about ‘caught in the act’ of grabbing, not DDoS.

This article merely stated a matter of fact as seen by the server’s log, no accusation was made as to WHY, merely gussess which TR is entitled to.

A hardware firewall NEED NOT be restricted to just blocking a DDoS attack, it can also block SYNC attack or port scanning attempts without shutting everyone out. Unwanted traffic are merely filtered and clean ones allowed through. Also, a DDoS attack NEED NOT be using spoofed IPs.

Being IT savvy as you claimed, you should know without me telling you.

A spider brought down TR? on Tue, 3rd Nov 2009 12:40 am 

Xisd, I think we can easily agree any DDOS with open IP is half-past-six. There’s no need to worry about them. But I will worry for TR if a lone spider can load up TR to cause their concern. TR needs to review its hardware.

ObviousMan on Tue, 3rd Nov 2009 12:49 am 

Nope. It is particularly hard to trace who is the source of a DDOS attack. In many cases, the IPs logged belong to innocent PCs that were directed to hit the target site.

The point is that it is not enough to pinpoint the IP. To even have a case, you have to prove that SPH was intentionally committing the act (and not some intruder sending a forged IP (eg. TR’s) to their network which causes their machines to reply to the targeted IP).

Without help in even ascertaining the facts, it is next to impossible to have any legal case.

XiSd Tay on Tue, 3rd Nov 2009 12:50 am 

A spider brought down TR, technically speaking yes but a poorly configured server can be brought down even by less than 100 malicious IPs during a DDoS.

Servers running cPanel usually has Apache installed as opposed to some other better alternatives like lighthttpd. Apache has a large footprint and is memory intensive with limits as to the maximum number of simultaneous connections it can handle at a go. If a spider sends 300 keepalives request at a go to access a site, that single spider from a single IP would have effectively caused a bottleneck effect blocking new connections till Apache has completed serving all these replies or timed the connection out, whichever is earlier. Hardware or no hardware, I have not seen servers with cPanel running Apache going past the 300 maxservers limit but I am no IT expert here and I stand corrected.

TR as I see it, is not concerned about the lone spider (admitted for the purpose of discussion) but merely curious as to why SPH is ‘interested’ in their site’s content dating back to 2008, I believe.

sheldon on Tue, 3rd Nov 2009 1:36 am 

if the grabbing was an orchestrated affair, meaning with planning and stuff, i find it hard to believe that they would willingly leave their ip trace for all to see.

either they are behaving like ‘real gangster not scared of police’ kind, or the real orchestrator is leading TR and its readers. why will have to depend on who the real orchestrator is.

put it this way, if the purpose of sph is to slow or bring down TR, i strain to think that it will do such a half ass job, considering it has the resources to do a better job. surely it is within their easy means to totally bring it down without being traced. i mean, do a proper ddos lah why just grab?

XiSd Tay on Tue, 3rd Nov 2009 2:17 am 

@sheldon, I do agree with your analogy.

Which is probably also the reason why TR’s admin called it grabbing and not an attack.

qussl3 on Tue, 3rd Nov 2009 2:24 am 

If the TR truly believes that this was an attack by its competitors but is worried about not getting a fair shake, consider going viral.

Use the social networks to spread the word and let public opinion decide who is right and wrong.

Something juicy like this likely to spread like wildfire if given the proper spark, the kindling is already there.

sheldon on Tue, 3rd Nov 2009 3:05 am 

indeed. grabbing sounds apt.

but surely they can do a much better job?

Anonymous on Tue, 3rd Nov 2009 4:05 am 

I’m not affiliated with SPH in any way, but I’m quite certain that this is their RedNano spider that you’re seeing causing the accesses.

If you check the User Agent header in your apache logs (not shown in the extract provided above), it should reflect that fact. RedNano is run by SPH Search Pte Ltd which of course is a division of SPH and would legitimately use an IP block assigned to SPH.

Looking at TR’s robots.txt, there’s no prohibition against any spider (whether it be Google’s, Yahoo’s, RedNano’s or even my own) from crawling your entire site except for two configuration type directories which have been masked out. robots.txt is the industry standard by which you allow or disallow site crawling. TR currently has crawling indicated as allowed and that’s exactly what SPH did.

Now, if you had prohibited SPH Search’s spider from crawling by making an explicit entry in your robots.txt directing it not to, and if it continued to do so, then I’d agree that you have a legitimate gripe against SPH and should hit them with the full force of the law. However, the way your site is configured now, what they did is perfectly acceptable.

Oh, one final word – you didn’t include timestamps in your logs or mention exactly how fast the site was being crawled. Any modern machine should be able to handle tens of hits per second – and even if your request volume exceeds that, a well configured server will degrade gracefully, and certainly not crash. Check your logs again and see just how fast SPH was crawling your content. If it was at anything faster than maybe 2 hits per second, then you’ve got a point, and they were likely causing a good dent in your overall access times. If it was slower than that, then they’re doing a responsible crawl.

I’m an avid TR reader, and was distressed to see an article presenting such a misinformed view. Excuse my rant…

Anonymous Coward; Cult of the common sense on Tue, 3rd Nov 2009 9:20 am 

@Anonymous/qussl3

The robots.txt file is in no way a legal binding agreement that robots should not crawl that particular area of the website. http://www.robotstxt.org/faq/legal.html for reference

Also, IANAL.

In my previous posts which for some reason isn’t showing up, I did mention that this article is technically unsound and TR needs to be less paranoid the moment someone does a HTTP get function to browse or index archives which are in *public* domain. The keyword here is *public* domain and a HTTP get is a legit HTTP request and it should not constitute overloading. This is pretty much the same (maybe got some legal differences lah, I don’t know) as a slashdot or a digg effect *IF* your server is being overloaded by multiple legit requests and personally, I don’t see it as a Distributed Denial of Service because it’s not really distributed if you consider it only from a single subnet where you can blacklist the entire subnet really. This really saying that the moment someone or maybe a web crawler that falls under any subnet which belongs to the government visits your website, you’re yelling murder and think it’s a hacking attempt.

Trust me when I say that if they were serious about blowing your server out of the water, it’s very possible and you probably wouldn’t even get the chance to be having this article up without upstream intervention of traffic scrubbing to filter the DDoS traffic from legit requests.

Also, a firewall doesn’t “strengthen” your server against multiple legit HTTP gets/post/delete or whatnots. A firewall matches traffic based on rules and generally, if it doesn’t match a permit rule, it gets denied when it reaches the end of the chain and as Anonymous 2 posts above me said, you have logs but you have no timestamps and that practically makes the logs worthless.

Qussl3, some of us have already pointed out why this article fails to be technically sound and it’ll not be in TR’s best interest to be publishing something like that and being made to seem as if they are pointing fingers at SPH for something “illegal” when it may be just a web crawler indexing data in public domain.

With all due respect to TR, more work needs to be put into QC’ing such an article before it hits public domain, it doesn’t look good. I enjoy the other articles, just not this one.

Anonymous Coward

PRrrr on Tue, 3rd Nov 2009 9:27 am 

Hi TR, you might want to use this chance to drum up awareness for this site. Good news or bad new is good PR.

XiSd Tay on Tue, 3rd Nov 2009 9:43 am 

@Anonymous, just so we are clear on the facts as presented and since admin is not IT savvvy compared to what I know (beginner stage), which is the main reason why I got engaged in this thread anyway. The aim is of course not to defend admin if he was wrong, but to state a mere fact.

One cannot blame you for your reasoning (which is sound and logical by the way), considering that you have no access to the logs or facts. Allow me to present some details for your consideration, you can then draw a fresh conclusion from there.

RedNano is owned and operated by SPH, that is TRUE. However, it is run by a divison of SPH, namely SPH SEARCH PTE LTD in Toa Payoh. Their servers (ie: rednano) is located at Expan Internet Data Centre, which uses the IP range 202.176.192.0 – 202.176.223.255.

RedNano is allocated the IP 202.176.218.20, as can be seen from http://www.domaintools.com/reverse-ip/?hostname=202.176.218.20

From the server logs and admin’s postings, the IP address doing the grabbing was 203.116.231.234, which belongs to STARHUB, worlds apart, if you ask me. http://whois.domaintools.com/203.116.231.234

So logic dictates that whatever or whoever doing the crawling or grabbing (whether legal or illegal not being an issue) is CERTAINLY NOT RedNano. Maybe SPH also has BlackNano or WhiteNano?

For security reason, I cannot post a copy of the long logs which was handed to me by admin when he sough advice but I can assure you that when TR’s SysAdmin said ‘furry of connections’, it wasn’t simply just a 1 connect per second which WOULD NOT have caused a degradation in a server’s performance no matter how badly configured the server was. It was in excess of 100 connects per minute.

Like I said many times, the crawling did cause a certain server load (as informed by TR’s sysadmin) but certainly DID NOT crash the server nor cause a significant degradation in the server’s performance because the new firewall they had was configured to drop and ban any IPs exceeding 60 connects per minute. The offending IP was banned less than 2 minutes into the act, according to their firewall log.

So we can safely conclude that it wasn’t RedNano, whatever or whoever it was, your good is as good as mine.

@Anonymous Coward, legality of the crawl is not in question unless it degrades the server’s performance substantially. Tr has acknowledged that their articles is PUBLIC DOMAIN and any Tom, Dick or Harry can view it. Their question is, why bother to GRAB? Anyway, it was merely reported that an incident had occured, no insinuation was made to anything else.

A legit HTTP request CAN FLOOD a server if it is intended to be malicious, please read my earlier reply with regards to Apache. DDoS is also a form of legit http request with a slight twist, ie: sending multiple request simultaneously without receiving a reply, technically. In this case, admin has stated clearly that IT WASN’T a DDoS, just a web grabber of sort.

@sheldon, you mentioned: but surely they can do a much better job?

Were you referring to SPH or TR?

FINALLY, to Anonymous, Anonymous Coward and those claiming that it was RedNano, can we please take RedNano out of the equation and continue this private conversation?

anon on Tue, 3rd Nov 2009 9:45 am 

I read with interest. Something above caught me.

Your (Temasek review’s) constant rhetoric against the establishment seems very indiscriminate now. You fail to realise there’re individual people in the establishment.

No doubt they may be PRs or foreigners which you so loathe (demonstrated by your many xenophobic articles), but given the statistics, they are more likely to be Singaporeans. Half of them serve NS too. You must know that, by targeting these faceless cold establishments, you’re actually also aiming at the hapless flesh and blood employees too, whether it be HDB or SPH.

This IP saga is an ultimate classic. I’m not acquainted with how a spider or grabber works, but I gather from above they have their legitimate uses. As machines, the creepy insects won’t feel anything. However, if it really happens to be a genuine reader, I’m sure he will feel much trepidation for having his IP revealed. Somemore, he’s using a work computer.

All the while, you’re anyhow throwing cluster bombs at the establishments. In this instance, you may well have thrown a grenade at a reader.

admin on Tue, 3rd Nov 2009 9:47 am 

Hi all,

Please read the article carefully. Nowhere did we accuse SPH of being responsible for the DDOS attack.

The article is deliberately drafted to leave out the technical jargons so that laymen can understand it.

That’s why we began the article by going down to the basics explaining DDOS and IP addresses.

The server log was much longer than what was printed here. Of course due to security reasons, we cannot reveal anything more than necessary.

The bare facts are presented as it is. As Xisdtay has pointed out, the IP address does not belong to Rednano and neither is it due to spiders. We leave you to reach your own conclusion.

The bottomline is: we hope that there will be no repeat of this whatever the reason is.

We cannot grow the readership if the site keeps crashing or is slowed down significantly to affect the reading now and then.

That’s why we want to send a message across especially to SPH that we are more than willing to email them all our files in xml format from 2008 till now to save them the trouble.

sheldon on Tue, 3rd Nov 2009 9:59 am 

xisdtay

sph.

admin on Tue, 3rd Nov 2009 10:17 am 

Hi anon,

We are not xenophobic and neither are we against foreigners and PRs. Do you know that our Managing Editor is a foreigner herself? And we have many writers who are PRs like Imran Ahmed. We are not anti-establishment either. We are focused only on issues.

As for the point you brought up, it is merely speculation. Spiders are routine visitors to any servers which allow them entry. They do not come suddenly in a flurry at one go.

XiSd Tay on Tue, 3rd Nov 2009 10:24 am 

@qussl3 and PRrr, TR is already an established site, there is no need for TR to have any more adverse publicity.

In this incident, TR merely stated a matter of fact, the conclusion is best left to the readers to draw for themselves.

@anon, would a genuine reader be reading 100 pages within 1 minute, dating back to archives in 2008? You tell me.

TR’s SysAdmin is an experienced network administrator from an offshore company, I trust he can tell the differences between a normal and conventional http request against a furry of http requests.

WatchSG on Tue, 3rd Nov 2009 11:17 am 

Admin,

Can your server sustain another similar incident? It is really frustrating to be logging in but to find your site down again. I think it happened at least a few times over the weekends?

I noticed that the site was constantly under upgrade or something like that for the past weeks. The site used to be good and stable previously leh.

Terrified on Tue, 3rd Nov 2009 11:47 am 

The main stream media has nothing new to write, so they are stealing news from new media. This is really shocking and their standard is so laughable.

Matt on Tue, 3rd Nov 2009 12:32 pm 

just a question : can you cite the computer misuse act when the server is hosted overseas to prevent political censorship?

jolly on Tue, 3rd Nov 2009 12:47 pm 

I guess the staff at SPH are only trying to survive in this competitive world by copying what others are doing and doing successfully. This is applicable across all government ministries and departments and also the private sector . borrow here and there just to keep their pay intact. HaHa

qussl3 on Tue, 3rd Nov 2009 1:53 pm 

@XiSd Tay

Instead of looking at this as adverse publicity (why is it adverse anyway), if the facts presented here are as such, this is an legitimate opportunity for TR to increase their profile.

All i suggest is to get the word out through unconventional channels rather than to depend on the MSM allowing your grievances to be aired.

If the site failures are directly a result of the ‘grabbing’ described here and not due to resource inadequacies on TR’s part then it is a legitimate issue (although it may still look like the TR is just incompetent).

alwin lai on Tue, 3rd Nov 2009 2:42 pm 

WP, i have idea. try go and write at Today as they are not part of SPH. They may report on this

Who is The David... And Who is The Goliath... THAT WILL Disipate???... on Tue, 3rd Nov 2009 3:09 pm 

QUOTING…
citizenofSG on Mon, 2nd Nov 2009 2:36 pm

Latest show in Singapore. TR vs SPH . David vs Goliath.
Nah I think the matter will be dissipated in the wind.
UNQUOTING…

As in the Biblical account of The Shephard BOY David… HE Killed The Giant Goliath that all in Israel feared!!!…

So… anaolgious to that account… TR should be DAVID and the other guy is the…???… YOUR GUESS IS AS GOOD AS MINE GUYS!!!…

As who FELL and DIED then!!!… The things is that those who have become ‘giants’ of sorts… never learn from the annals of history… which has the perfect habit of repeating itself in human nature… that ‘giants’ made… and then made ‘giants’ ALWAYS FALL!!! Amen!!! As that IS the way of a divine God!!!

bornloser on Tue, 3rd Nov 2009 3:21 pm 

Just when Shanmugam told the Americans we are under the rule of law, SPH went right out to break the law! Will they be subject to the law?

btan on Tue, 3rd Nov 2009 5:43 pm 

We can only speculate who is the mastermind behind the DDOS attack. Based on motive, there are three groups of suspects.

1.) PAP or their supporters/subsidiaries. Based on motive, since this site often exposed the weakness and fallacies of PAP and has a lot of strong anti-PAP readers, they are an obvious group to launch such and attack to weaken TR.

2.) Business competitors of TR. This could either be MSM or other blog or news sites like TOC. What more to raise your own readership by bringing down the site of your competitor. Political content wise, TR is number 1 in terms of online readership.

3.) Hackers/Script Kiddies. These are people who has not other motive other than the thrill of proving their e-peens by bringing down a website, just for the fun of it.

In any case, TR should launch a police report just for the DDOS attack and let the computer division do the job. TR should also engage the advise of a lawyer to see what cause of action they can do in the event the culprit is identified.

George on Tue, 3rd Nov 2009 6:36 pm 

Has an offence been disclosed by this. Why not put in a police report. That should be interesting.

3rd Class citizen on Tue, 3rd Nov 2009 8:02 pm 

Now I know about this Act:

Section 7 of the Computer Misuse Act (CAP 50A):

7. —(1) Any person who, knowingly and without authority or lawful excuse —

(a) interferes with, or interrupts or obstructs the lawful use of, a computer; or

(b) impedes or prevents access to, or impairs the usefulness or effectiveness of, any program or data stored in a computer,
shall be guilty of an offence and shall be liable on conviction to a fine not exceeding $10,000 or to imprisonment for a term not exceeding 3 years or to both and, in the case of a second or subsequent conviction, to a fine not exceeding $20,000 or to imprisonment for a term not exceeding 5 years or to both.

A spider brought down TR? on Wed, 4th Nov 2009 1:11 am 

Xisd, you are able to change your comments. You added worries about cpanel. But we won’t go into specifics like limitations and performance.

In general, the setup is sized according to expectations. If one server can’t do it, there can be 2 or even a farm. If not, there are other ways to manage things gracefully instead of a full crash.

For anything more serious than a personal homepage, it’s mandatory to consider common redundancies and malicious behaviours. Don’t take things for granted for a lifeline.

These are general guidelines. It’s easier said than done, for clients usually don’t know what they want.

So, if it’s really that a spider (a grabber is also a spider) stress the site too much, then better review the setup or bring in solutions to manage it.

A typical DDOS is the one that shut down twitter. For a “DDOS” defeated by an entry in the firewall, the people should just laugh really hard at the “attacker” instead of worrying too much.

halo on Wed, 4th Nov 2009 6:15 am 

Stupid admin u r creating war here…

it is robot from sph, not human

should be their search engine rednano

stupid idiot admin..please resign

XiSd Tay on Wed, 4th Nov 2009 8:22 am 

Halo, you should really leanr to read other’s reply before calling someone stupid.

RedNano is owned and operated by SPH, that is TRUE. However, it is run by a divison of SPH, namely SPH SEARCH PTE LTD in Toa Payoh. Their servers (ie: rednano) is located at Expan Internet Data Centre, which uses the IP range 202.176.192.0 – 202.176.223.255.

RedNano is allocated the IP 202.176.218.20, as can be seen from http://www.domaintools.com/reverse-ip/?hostname=202.176.218.20

From the server logs and admin’s postings, the IP address doing the grabbing was 203.116.231.234, which belongs to STARHUB, worlds apart, if you ask me. http://whois.domaintools.com/203.116.231.234

WatchSG on Wed, 4th Nov 2009 9:32 am 

Err…sorry but am I missing something? what is the point of making so much noise here but not doing anything about it? If not doing anything mean there is no cause for concern?

So what if SPH grab things from TR? It is not an attack rite? So no offence? TR is an open portal mah…I suppose this is part and parcel of things then since it is public?

ExPoLiCe on Wed, 4th Nov 2009 10:25 am 

@XiSd Tay,

You are being modest. Your IT knowledge cannot be at beginner level. If you are, then I cannot find a term for my level then. Novice?

You said that Rednano is given the IP address 202.176.218.20 and from my very low level understanding, this would only mean that the webserver, hosting http://www.rednano.com.sg is at this IP address. Can the spiders, working for rednano be coming from another IP address like the one you captured?

Awaiting your enlightenment.

Robox on Wed, 4th Nov 2009 10:59 am 

TR Admin, I’m really confused as to what might be happening here. But one thing I can say that might be useful is that if you manage to determine that there is a guilty party here and possibly a well connected one that is responsible for all this, I suggest that the best way to deal with this as legally as possible is the following:

You are looking for justice and the meting out of justice has these aims:

1. retribution or restitution;
2. deterence; and/or,
3. rehabilation.

If a police case or other established legal avenues are open to you, by all means go for it. But the problem is – and always has been your anonimity, which btw is a NOT a complaint on my part.

That being the case, retribution or restitution are avenues that closed to you.

Then go on to the next best thing in seeking justice: deterrence and/or rehabilitation.

The thing that any guilty party fears most is exposure and publicity. If you are able to determine guilt, then publicize it. Recently, RSF has come into the news again, and one of the many criteria that they use to determine the level of freedom of the press is what happens on the internet. Report any relevant findings to them, and make matters worse for the Singapoe government if they are involved in this.

Robox on Wed, 4th Nov 2009 11:03 am 

Sorry, I neglected to say that by taking the action of negative publicity against the guilty party or parties, you would actually be preventing the recurrence (ie. deterrence and therefore rehabilitation) of these events happening again.

I am counting on you to remain accessible at elections time when you can expect a huge surge in usership. Act now, and we will continue to support those who support us.

ExPoLiCe on Wed, 4th Nov 2009 11:13 am 

Lodging a police report is not going to help.

Checked with friends who are familiar with the CMA and according to them, case is pretty weak.

admin on Wed, 4th Nov 2009 11:29 am 

Hi Robox,

Thanks for your suggestion. We will not be pursuing any further action as of now.

The purpose of this article is just to send a warning to those who harbor the intention to bring our site down for whatever reasons that we are prepared and they will get caught one day if they push their luck too far. As you can see from the above discussions, there are some with dubious motives who are deliberately baiting us to reveal more information about our server.

TR bears no enmity towards any sites, bloggers or gamers in cyberspace. All we want is to be left alone to pursue our own interests. We do not care a hoot about what happen elsewhere. It is not that we cannot trace the culprit, but we do not want to spend unncessary money on it when it can be put to better use. However, if they peeve us off too much, we will not hesitate to spend and really spend to flush them out.

It may be too late for us to prepare for the next election now, but come the election after next, we may have a more formidable structure in place.

Rome isn’t built in a day. Right now, we must be patient and wait.

Robox on Wed, 4th Nov 2009 12:45 pm 

Thanks for your reply Admin. I really hope the best for you. I trust you know what you are doing because I’m really no tech guy.

btan on Wed, 4th Nov 2009 1:33 pm 

Hi TR,

Next election is one or two years away, why will you not be ready? I am sure you can be prepared within 1 or 2 years right?

For all we know, the coming election may be a significant milestone for Singapore. If you are not ready to cover it, you may miss the golden opportunity. I hope you and TOC are around in this coming election since both of you combined has the highest online readership.

In the previous election, we only had sites like yawning bread and some odds and ends sites with no centralised coverage, yet still have enough coverage to cover the important stuff.

With centralised coverage, more information will be available. Chief among are the profiles of candidates which MSM will definitely give prominence to PAP, hence we need alternative news sites to balance off any coverage not done by MSM and to counter any negative slants as well.

Vote of Opposition on Wed, 4th Nov 2009 4:05 pm 

Admin,

If you have sufficient evident, then you should sue the SPH for their cyber attacks.

But please do not do that if something liked NKF happens will happen!

Sinkapore on Wed, 4th Nov 2009 10:09 pm 

@ExPoLiCe, you missed the keyword, its RedNano’s Servers, not just the site or the domain name.

Surely the site can be mirrored all over the world but not in the case of RedNano, their servers are all in one DC, so no question of them using another DC for their spiders.

As to how I know, its priviledged information

Sinkapore on Fri, 6th Nov 2009 12:55 pm 

Vote of Opposition, all of you are missing the point here. What TR said was there SPH’s IP (ie: someone or something) was caught grabbing TR’s site on the date in question. There was no accusation of any attacks or malicious activities, merely that grabbing occured.

Police report, legal suit is out of the question as TR’s articles are all in Public Domain meant for public reading, but not grabbing.

Bottomline is, SPH being ‘heavilly backed and connected’ will and can come up with all sorts of ‘experts’ to testify that their servers were down on that day for maintainance, upgrading or that day server farm all kenna lock down, etc etc etc.

Its basically their word against TR’s admin. Even if TR manages to get their DC Chief Engineer to sign a written statement that the IP was indeed reverse traced to SPH servers, SPH will probably find some forensic experts or someone grossly over qualified to debate that.

This article is for information only and readers should draw their own conclusion as to which party to believe.

Geoffrey Pereira on Fri, 6th Nov 2009 1:36 pm 

Sorry, I accidentally ommitted a “not” my post should read

“SPH did not attack TR web servers or grab content as claimed. See here: http://blogs.straitstimes.com/2009/11/6/attack-on-temasek-review-site-not-sph”

Grateful if you could post the corrected version

Geoff

Anonymous Coward; Cult of the common sense on Fri, 6th Nov 2009 1:56 pm 

@Sinkapore

As stated by some of the readers, myself included, the article is sorely lacking in technical evidence and if TR is *really* sure that DDoS or overloading the server (Point of contention) has taken place, both parties are free to release the logs of the servers and relevant data logs from Orion/Cacti/ and I’m sure that there are other neutral 3rd parties can dissect those for you. There’s really no reason to be “presuming” something that may or may not have taken place.

Anonymous Coward

Vote of Opposition on Fri, 6th Nov 2009 2:08 pm 

So grabbing and the spider bots from search engines are different? I am no IT person

xingrencha on Fri, 6th Nov 2009 3:51 pm 

I don’t know y Geoff P even bothered to reply to the ridiculous charges by TR… which is opaque abt its funding and censors comments not flattering to it. Talk of a dictatorship in the flesh n TR is it!

Sinkapore on Fri, 6th Nov 2009 4:12 pm 

@Anonymous Coward, why are all of you missing the core point here? TR merely stated that SPH’s IP were caught grabbing TR’s website. NOTHING ELSE were insinuated and no foul play was suggested.

@xingrencha, WHY SHOULD TR releases its books since it is a PRIVATE Blog?

As for censorship, which BLOG IN THE WORLD doesn’t censor inappropriate comments and spam? Shitty Times?

ExPoLiCe on Fri, 6th Nov 2009 4:33 pm 

@Sinkapore,

From a neutral view point, I do feel that the initial article by TR, though did not allege that SPH is responsible for the DDoS, but had did enough to make people feel that SPH is doing or planning something malicious against TR.

The explanation on the DDoS and the citing of the Sec 7 of the CMA is uncalled for since they are irrelevant here – ie, SPH is not behind the DDoS and grabbing of “public content” would not tentamount to an offence under CMA (unless it was done in such a ferocious manner that it brings the server down).

I feel that it is the perception that TR had created that invited an response from SPH, not the allegation itself.

Sinkapore on Fri, 6th Nov 2009 4:47 pm 

@ExPoLiCe, the article is meant to be read as it is. The writer cannot be blamed if they choose to infer anything from reading it although it did suggest that SPH is ‘doing something’. as to what ’somnething’ is, your guess is as good as mine.

With regards to the response by SPH, its expected and I am not totally surprised they did.

ExPoLiCe on Fri, 6th Nov 2009 4:59 pm 

@Sinkapore,

I beg to differ.

If the writer merely reported the finding as it is and readers inferred based on their imagination, then I would agree with you that yes, the writer should not be held accountable.

However, if the writer put in irrelevant and subjective elements into the article and that led to the inference from the readers, then you cannot say that the writer cannot be blamed.

admin on Fri, 6th Nov 2009 5:02 pm 

Hi Geoffrey Pereira,

First, before we began, we must thank you for helping us publicize our site.

We are sorry that you were “arrowed” by your superiors to draft a reply to us and we have already published a point-by-point rebuttal which unfortunately put you in a bad light.

The administrators of this site are very reasonable people and it was not our intention to make things difficult for you.

None of us are technical people and it will be unfair for us to continue the exchange here.

What we propose is very simple:

1. Get the system administrator of SPH to contact our hosting company RTG (Asia) Network and asked for our full server log.

2. Investigate why our log showed SPH IP addresses “grabbing” our content at the stated time frame on 31 October 2009, 2200 hours to 1 November 2009, 0100 hours.

If the system administrator at the company somehow made a mistake or gave us wrong information, request them to publish it on its site.

We will follow suit with an unreserved apology immediately.

However, if it is indeed true that the perpetuator is a SPH staff, we hope SPH can give us an explanation of what really happened.

Sinkapore on Fri, 6th Nov 2009 5:12 pm 

Admin, I suggest the writer contact China Telecom (owner of the DC) Chief Engineer to sort things out if they feel they have a strong case for rebuttal.

On a technical level, the experts at SPH can pit their expertise with those from China Telecoms, both corporate GIANTS in their own rights.

But then, why bother? Just insist that the IP was spoofed, end of story, it would take a lot of experts and finances to do a forensic investigation on the electronic signatures left behind at China’s DC level right through to the mainstream provider linking China and Sinkapore. Definitely a never ending story with no sure win situation.

Maybe SPH or Temasek Holdings can buy out China Telecoms and get their CEO or Chief Engineer to issue a public statement disputing TR’s facts, then sure to win. Afterall, not a possibility to be dismissed considering that ‘face’ is at stake.

BTW, what position is Geoffrey Pereira holding at SPH? System Administrator of their LAN or just a spin doctor, err.. I meant journalist kenna arrowed?

ExPoLiCe on Fri, 6th Nov 2009 5:19 pm 

@admin,

Kudos to you for taking swift action to end this squabbles and the readiness to apologise should TR really make a mistake.

To show your sincerity, you should email Mr Pereira the contact info of your system admin at RTG Asia to facilitate the verification process.

@Sinkapore,

I do not think that it is right to refer SPH to China Telecom. TR paid RTG Asia for the hosting service, not China Telcom. Am I right?

rolleyes on Fri, 6th Nov 2009 5:23 pm 

Admin, you need to clarify. Are you more worried about the grabbing or the “DDOS”? If you’re worried about the “DDOS”, you better go after the attacker.

If you’re worried about the grabbing, then you are just whining. Adn guess what, you’ve already told them the secret about how you block the grabbing. Anyone can easily design or configure any grabber to go around that block. That’s a fact.

I suggest this. You can discuss with your IT people. Publish your articles in encoded form and use a java applet which can decode your articles to display in the browser. Good idea?

Sinkapore on Fri, 6th Nov 2009 5:29 pm 

@ExPoLiCe, I was just trying to be funny

The company’s site is located at http://www.rtg.asia and China Telecoms is at http://www.chinatelecom.com.cn

rolleyes on Fri, 6th Nov 2009 5:31 pm 

By the way, once you are able to stop grabbing, you also stop indexing too. Ask your IT what that is about.

ExPoLiCe on Fri, 6th Nov 2009 5:33 pm 

@Sinkapore,

Actually, the simpliest way is to ask XiSd Tay for the logs.

I am also trying to be funny here.

Sinkapore on Fri, 6th Nov 2009 5:52 pm 

@ExPoLiCe, Xisd Tay HAS the server log, mrtg bandwidth graph and firewall log because the admin of TR had asked for his opinion prior to publishing the initial article.

However, since the position of Xisd Tay is ’special’, so anything he says also ‘2 head not reach shore’. Correct?

Furthermore, SPH is a well-connected CORPORATE GIANT with very very very strong backing, who the fiack is Xisd Tay?

Yeeloong on Fri, 6th Nov 2009 7:59 pm 

“and Brutus is an honorable man”….

Sinkapore, unfortunately, if they start by leading with the definition of DDOS and go on to talk about an SPH IP being logged as accessing your website,I think it is rather myopic to keep pounding that TR did not accuse SPH of launching an attack please. Just as Mark Anthony in no way suggested that Brutus had anything to do with Caeser’s assassination, but repeatedly begged the questio of Brutus’s honour instead… This would be called a leading question in court.

Sinkapore on Fri, 6th Nov 2009 8:37 pm 

Yeelong, you are entitled to form your own opinion, no problem with me there.

Leading or not leading, we are not in court.

disgusted on Sat, 7th Nov 2009 1:49 am 

Dear editors of TR,

Thank you for not publishing my comment. Now I can spread the word, with conviction, that TR is guilty of the censorship that they accuse others of. Not that I can say I’m surprised.

Spaceman on Mon, 9th Nov 2009 8:44 am 

Sue & sue SPH, TR can put the money to good use.

aceman on Mon, 9th Nov 2009 8:47 am 

Yep, sue for defamation, afflamation, whatever-mation, follow the leader, mah!