Google Bot and yahoo bot IP's

Ph1ngering · Joined: 13 Feb 2008 Posts: 27

Just curious if any one has a good list of what IPs google and yahoo uses for there crawls..

I have one crawling my site 24x7 its seems, but i don't think its the deep crawl bot, its the daily fresh bot.. Just curious if any one knows the different ip ranges between the fresh bot and the deep bot.. (for lack of better terms)

66.249.73.57 google and 74.6.28.42 yahoo are both crawling me now..

Thanks
--Craig

Ph1ngering · Joined: 13 Feb 2008 Posts: 27

74.6.28.78 another yahoo crawler

74.6.27.74 - another yahoo crawler

Both have been on the site for 4 hours crawling..

Googles 66.249.73.57 has been crawling since the last post..

Any one know the ip's for the deep crawler?

Peter77

Google ip's

66.249.65.133
66.249.67.72
66.249.72.137

Yahoo ip's
72.30.179.200
72.30.179.204
72.30.179.212
72.30.179.215
72.30.179.217
72.30.179.218
72.30.179.225
72.30.179.230
72.30.179.233
72.30.179.235
74.6.19.107
74.6.19.109
74.6.19.141
74.6.19.142
74.6.19.151
74.6.19.154
74.6.19.159
74.6.19.160
74.6.19.165
74.6.19.198
74.6.19.200
74.6.19.202
74.6.19.210
74.6.19.214
74.6.19.218
74.6.19.221
74.6.19.222
74.6.19.223
74.6.19.232
74.6.19.233
74.6.19.236
74.6.19.237
74.6.19.75
74.6.19.80
74.6.19.82
74.6.20.101
74.6.20.103
74.6.20.114
74.6.20.116
74.6.20.12
74.6.20.136
74.6.20.14
74.6.20.147
74.6.20.153
74.6.20.16
74.6.20.161
74.6.20.166
74.6.20.17
74.6.20.19
74.6.20.198
74.6.20.203
74.6.20.207
74.6.20.21
74.6.20.211
74.6.20.212
74.6.20.214
74.6.20.219
74.6.20.222
74.6.20.224
74.6.20.229
74.6.20.23
74.6.20.26
74.6.20.32
74.6.20.36
74.6.20.39
74.6.20.41
74.6.20.70
74.6.20.71
74.6.20.75
74.6.22.13
74.6.22.15
74.6.22.18
74.6.22.19
74.6.22.205
74.6.22.21
74.6.22.211
74.6.22.22
74.6.22.223
74.6.22.224
74.6.22.225
74.6.22.226
74.6.22.229
74.6.22.230
74.6.22.233
74.6.22.31
74.6.22.37
74.6.22.39
74.6.22.41
74.6.22.44
74.6.22.49
74.6.23.105
74.6.23.108
74.6.23.110
74.6.23.117
74.6.23.121
74.6.23.122
74.6.23.139
74.6.23.142
74.6.23.147
74.6.23.15
74.6.23.150
74.6.23.159
74.6.23.164
74.6.23.199
74.6.23.203
74.6.23.204
74.6.23.210
74.6.23.213
74.6.23.216
74.6.23.22
74.6.23.220
74.6.23.221
74.6.23.222
74.6.23.231
74.6.23.233
74.6.23.236
74.6.23.26
74.6.23.29
74.6.23.38
74.6.23.70
74.6.23.82
74.6.24.10
74.6.24.102
74.6.24.104
74.6.24.105
74.6.24.107
74.6.24.112
74.6.24.123
74.6.24.13
74.6.24.134
74.6.24.137
74.6.24.140
74.6.24.142
74.6.24.145
74.6.24.146
74.6.24.152
74.6.24.153
74.6.24.169
74.6.24.18
74.6.24.198
74.6.24.20
74.6.24.203
74.6.24.209
74.6.24.210
74.6.24.214
74.6.24.217
74.6.24.219
74.6.24.22
74.6.24.221
74.6.24.222
74.6.24.223
74.6.24.226
74.6.24.234
74.6.24.26
74.6.24.28
74.6.24.30
74.6.24.34
74.6.24.36
74.6.24.37
74.6.24.45
74.6.24.46
74.6.24.70
74.6.24.72
74.6.24.82
74.6.25.103
74.6.25.104
74.6.25.117
74.6.25.118
74.6.25.120
74.6.25.123
74.6.25.124
74.6.25.137
74.6.25.139
74.6.25.143
74.6.25.147
74.6.25.148
74.6.25.15
74.6.25.160
74.6.25.161
74.6.25.172
74.6.25.19
74.6.25.199
74.6.25.200
74.6.25.205
74.6.25.209
74.6.25.211
74.6.25.216
74.6.25.217
74.6.25.220
74.6.25.223
74.6.25.24
74.6.25.31
74.6.25.32
74.6.25.36
74.6.25.70
74.6.25.74
74.6.25.75
74.6.25.77
74.6.25.81
74.6.25.82
74.6.26.103
74.6.26.105
74.6.26.114
74.6.26.137
74.6.26.139
74.6.26.140
74.6.26.143
74.6.26.144
74.6.26.145
74.6.26.154
74.6.26.156
74.6.26.16
74.6.26.160
74.6.26.163
74.6.26.165
74.6.26.166
74.6.26.199
74.6.26.207
74.6.26.220
74.6.26.221
74.6.26.228
74.6.26.231
74.6.26.232
74.6.26.31
74.6.26.38
74.6.26.39
74.6.26.72
74.6.26.82
74.6.27.103
74.6.27.123
74.6.27.14
74.6.27.16
74.6.27.31
74.6.27.34
74.6.27.35
74.6.27.47
74.6.27.72
74.6.28.101
74.6.28.109
74.6.28.11
74.6.28.121
74.6.28.134
74.6.28.138
74.6.28.139
74.6.28.149
74.6.28.150
74.6.28.160
74.6.28.162
74.6.28.163
74.6.28.164
74.6.28.170
74.6.28.173
74.6.28.18
74.6.28.200
74.6.28.206
74.6.28.21
74.6.28.212
74.6.28.214
74.6.28.215
74.6.28.216
74.6.28.217
74.6.28.218
74.6.28.22
74.6.28.221
74.6.28.224
74.6.28.229
74.6.28.237
74.6.28.25
74.6.28.31
74.6.28.36
74.6.28.39
74.6.28.47
74.6.28.73
74.6.28.76
74.6.28.78
74.6.29.13
74.6.29.30
74.6.29.37
74.6.29.38
74.6.29.40
74.6.29.42
74.6.29.43
74.6.29.49
74.6.7.101
74.6.7.112
74.6.7.122
74.6.7.124
74.6.7.126
74.6.7.135
74.6.7.138
74.6.7.139
74.6.7.140
74.6.7.146
74.6.7.151
74.6.7.154
74.6.7.166
74.6.7.169
74.6.7.173
74.6.7.71
74.6.7.74
74.6.7.81

Ph1ngering · Joined: 13 Feb 2008 Posts: 27

nice list, but do you know which ones are the daily quick crawl and which is the deep crawl?

Peter77 · Posted: Thu Feb 28, 2008 2:32 am Post subject: Re: Google Bot and yahoo bot IP's

My guess is the quick crawler... I might be wrong, but I am not 100% for sure.
Each day those are the ip's that show up on my stats and the ip's have been marked to have visisted 1 to 4 times in one 24 hour day.

dcz · Posted: Sat Mar 01, 2008 4:37 pm Post subject: Re: Google Bot and yahoo bot IP's

Ph1ngering · Joined: 13 Feb 2008 Posts: 27

Just something I had saw some where on the internet.. But it has said that monthly google will deeply crawl your site vrs just looking for one off updates.

Which would make sense to me to do it this way and would explain why it takes things that were index'd months ago so long to be updated again.

dcz · Posted: Wed Mar 12, 2008 2:10 pm Post subject: Re: Google Bot and yahoo bot IP's

Well, this is for sure, but it's rather related to the way Google determine and schedules crawling, I don't think that there is a distinction in between several bots to do so. And it would not make a difference in case it would, since it all goes down to a crawling list with priorities.

++