php bot hakkında yardım!

10-02-2014, 17:23:41

#1

~~hafsel~~

Üyeliği durduruldu

merhaba...
Php ile bot yapmayı öğreniyorum takıldığım bir kısım var bu konuda bilgili arkadaşlardan yardım istiyorum..Şimdi derdimi örnek vererek anlatayım site.com/sayfa-1 diye bir sayfa var ben bundaki içerikleri alıyorum istediğim gibi değiştirip yazdırıyorum burda sorun yok.ama bunu yalnızca bir sayfa için yapabiliyorum...yapmak istediğim şey bot ile karşı siteden devam eden sayfalarıda almak..sayfa-1, sayfa-2 ,sayfa-3 .... Sayfa-x gibi otomatik devam edip içerik almasını istiyorum... Yardımlarınızı bekliyorum
10-02-2014, 18:06:01

#2

ruinded

botla veri çektiğin sayfada "sonraki" gibi bir link varsa sonraki sayfanın url'sini alabilirsin.

10-02-2014, 22:40:59

saintx

Kimlik doğrulama veya yönetimden onay bekliyor.

@hafsel; mantığı anlamanız için ufak bi' bot yazdım. Takıldığınız kısımlar olursa beni mesajınıza etiketleyip sorularınızı yöneltebilirsiniz.

<?php
	
	date_default_timezone_set("Europe/Istanbul");
	
	define("BOT_BASE_URL", "http://www.webrazzi.com/");
	
	function bot_curl_get_content($url)
	{
		$handle = curl_init();
		
		curl_setopt($handle, CURLOPT_URL, $url);
		
		curl_setopt($handle, CURLOPT_BINARYTRANSFER, true);
		curl_setopt($handle, CURLOPT_RETURNTRANSFER, true);
		
		curl_setopt($handle, CURLOPT_SSL_VERIFYPEER, false);
		
		curl_setopt($handle, CURLOPT_TIMEOUT, 20);
		
		$response = curl_exec($handle);
		
		curl_close($handle);
		
		return $response;
	}
	
	function bot_get_categories()
	{
		$response = bot_curl_get_content(BOT_BASE_URL);
		
		$list = preg_match(
			"#\<h(1|2|3|4|5|6)\>Kategoriler\<\/h(1|2|3|4|5|6)\>.*?\<ul\>(.*?)\<\/ul\>#si",
			$response,
			$matches
		) ? end($matches) : null;
		
		$items = preg_match_all("#href\=\"(.*?)\".*?\>(.*?)\<\/a\>#si", $list, $matches) ? $matches : null;
		
		$response = array();
		
		foreach(end($items) as $key => $value)
		{
			$response[] = (object) array("url" => $items[1][$key], "title" => html_entity_decode($items[2][$key], ENT_QUOTES, "UTF-8"));
		}
		
		return $response;
	}
	
	function bot_build_category_url($category, $page_number)
	{
		return $page_number <= 1 ? $category : rtrim($category, "/") . "/page/" . $page_number . "/";
	}
	
	function bot_get_category_posts($category, $page_number)
	{
		$url = bot_build_category_url($category, $page_number);
		
		$response = bot_curl_get_content($url);
		
		$items = preg_match_all("#class\=\"post\".*?\<h2\>\<a.*?href\=\"(.*?)\".*?\>(.*?)\<\/a\>#si", $response, $matches) ? $matches : null;
		
		$response = array();
		
		foreach(end($items) as $key => $value)
		{
			$response[] = (object) array("url" => $items[1][$key], "title" => html_entity_decode($items[2][$key], ENT_QUOTES, "UTF-8"));
		}
		
		return $response;
	}
	
	function bot_post_details($post_urls)
	{
		$response = array();
		
		foreach($post_urls as $post_url)
		{
			$post_response = bot_curl_get_content($post_url);
			
			$post_title = preg_match(
				"#name\=\"twitter\:title\".*?content\=\"(.*?)\"#si",
				$post_response,
				$matches
			) ? end($matches) : null;
			
			$post_description = preg_match(
				"#name\=\"twitter\:description\".*?content\=\"(.*?)\"#si",
				$post_response,
				$matches
			) ? end($matches) : null;
			
			$post_keywords = preg_match(
				"#name\=\"keywords\".*?content\=\"(.*?)\"#si",
				$post_response,
				$matches
			) ? implode(", ", array_map("trim", explode(",", end($matches)))) : null;
			
			$response[] = (object) array(
				"title" => html_entity_decode($post_title, ENT_QUOTES, "UTF-8"),
				"description" => html_entity_decode($post_description, ENT_QUOTES, "UTF-8"),
				"keywords" => html_entity_decode($post_keywords, ENT_QUOTES, "UTF-8")
			);
		}
		
		return $response;
	}
	
?><!DOCTYPE html>
<html class="no-js">
	<head>
		<meta charset="UTF-8" />
		<meta name="author" content="Ogün Karakuş" />
		<title>@imsaintx</title>
		<link rel="author" type="text/plain" href="http://saintx.net/humans.txt" />
		<link rel="icon" type="image/x-icon" href="http://saintx.net/storage/assets/images/favicon.ico" />
		<link rel="shortcut icon" type="image/x-icon" href="http://saintx.net/storage/assets/images/favicon.ico" />
		<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/modernizr/2.7.1/modernizr.min.js"></script>
		<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/jquery/1.11.0/jquery.min.js"></script>
		<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.1.0/js/bootstrap.min.js"></script>
		<script type="text/javascript">
			$(function() {
				Modernizr.load([
					"https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.1.0/css/bootstrap.min.css",
					"https://cdnjs.cloudflare.com/ajax/libs/font-awesome/3.2.1/css/font-awesome.min.css"
				]);
				
				$("body").css("padding", "1em");
				
				$("label, select, input[type=\"checkbox\"]").css("cursor", "pointer");
				
				$("h1.panel-title a").attr("target", "_blank").attr("title", document.title).append(document.title);
				
				$("span").tooltip();
				
				$("a").tooltip({placement: "right"});
				
				$(".full-width").css("width", "100%");
			});
		</script>
	</head>
	<body>
		<form method="post" class="panel panel-default form-horizontal">
			<div class="panel-heading">
				<h1 class="panel-title">
					<i class="icon icon-globe"></i>&nbsp;<a href="http://saintx.net/"></a>
				</h1>
			</div><?php if(empty($_POST)): echo PHP_EOL; ?>
			<div class="panel-body">
				<div class="form-group">
					<label for="bot_category" class="col-sm-1 control-label">
						<span class="label label-default">Kategoriler:</span>
					</label>
					<div class="col-sm-11">
						<select id="bot_category" name="bot_category" class="form-control">
							<?php $values = ""; foreach(bot_get_categories() as $category): $values .= sprintf("<option value=\"%s\">%s</option>\n\t\t\t\t\t\t\t", $category->url, $category->title); endforeach; echo trim($values) . PHP_EOL; ?>
						</select>
					</div>
				</div>
				<div class="form-group">
					<label for="bot_page_number" class="col-sm-1 control-label">
						<span class="label label-default" data-toggle="tooltip" title="Sayfa Numarası">Sayfa:</span>
					</label>
					<div class="col-sm-11">
						<select id="bot_page_number" name="bot_page_number" class="form-control">
							<?php $values = ""; foreach(range(1, 250) as $page_number): $values .= sprintf("<option value=\"%s\">%s</option>\n\t\t\t\t\t\t\t", $page_number, $page_number); endforeach; echo trim($values) . PHP_EOL; ?>
						</select>
					</div>
				</div>
			</div>
			<div class="panel-footer">
				<button type="submit" class="btn btn-default full-width">Öğeleri Getir</button>
			</div>
		</form>
<?php elseif(isset($_POST["bot_category"]) && isset($_POST["bot_page_number"])): echo PHP_EOL; $posts = bot_get_category_posts($_POST["bot_category"], $_POST["bot_page_number"]); ?>
			<div class="panel-body">
				<?php foreach($posts as $post): ?><div class="form-group">
					<label class="col-sm-1 control-label">
						<input type="checkbox" name="bot_posts[]" value="<?=$post->url;?>" />
					</label>
					<div class="col-sm-11">
						<span class="form-control">
							<i class="icon icon-external-link"></i>
							<a href="<?=$post->url;?>" target="_blank" data-toggle="tooltip" title="<?=$post->title;?>"><?=$post->title;?></a>
						</span>
					</div>
				</div>
			<?php endforeach; ?></div>
			<div class="panel-footer">
				<button type="submit" class="btn btn-default full-width">Öğeleri Getir</button>
			</div>
		</form>
<?php elseif(isset($_POST["bot_posts"])): echo PHP_EOL; $details = bot_post_details($_POST["bot_posts"]); ?>
			<div class="panel-body">
				<?php $last_key = count($details) - 1; foreach($details as $key => $post): ?><div class="form-group">
					<label for="post_title_<?=$key;?>" class="col-sm-1 control-label">
						<span class="label label-default">Yazı Başlığı:</span>
					</label>
					<div class="col-sm-11">
						<input type="text" id="post_title_<?=$key;?>" class="form-control" name="post_title" value="<?=$post->title;?>" />
					</div>
				</div>
				<div class="form-group">
					<label for="post_description_<?=$key;?>" class="col-sm-1 control-label">
						<span class="label label-default">Yazı Açıklaması:</span>
					</label>
					<div class="col-sm-11">
						<textarea id="post_description_<?=$key;?>" class="form-control" name="post_description" rows="5"><?=$post->description;?></textarea>
					</div>
				</div>
				<div class="form-group">
					<label for="post_keywords_<?=$key;?>" class="col-sm-1 control-label">
						<span class="label label-default">Yazı Etiketleri:</span>
					</label>
					<div class="col-sm-11">
						<input type="text" id="post_keywords_<?=$key;?>" class="form-control" name="post_keywords" value="<?=$post->keywords;?>" />
					</div>
				</div>
			<?php if($key != $last_key): echo "<hr />"; endif; endforeach; ?></div>
			<div class="panel-footer">
				<button type="submit" class="btn btn-default full-width">Öğeleri Getir</button>
			</div>
		</form>
<?php endif; ?>
		<footer class="text-center">
			<small>Copyright &copy; 2012 - <?=date("Y");?>, <a href="http://saintx.net/" target="_blank">Ogün Karakuş</a></small>
		</footer>
	</body>
</html>

@pastebin

11-02-2014, 01:18:01

~~hafsel~~

Üyeliği durduruldu

@saintx;

saintx adlı üyeden alıntı: mesajı görüntüle

@hafsel; mantığı anlamanız için ufak bi' bot yazdım. Takıldığınız kısımlar olursa beni mesajınıza etiketleyip sorularınızı yöneltebilirsiniz.

<?php
	
	date_default_timezone_set("Europe/Istanbul");
	
	define("BOT_BASE_URL", "http://www.webrazzi.com/");
	
	function bot_curl_get_content($url)
	{
		$handle = curl_init();
		
		curl_setopt($handle, CURLOPT_URL, $url);
		
		curl_setopt($handle, CURLOPT_BINARYTRANSFER, true);
		curl_setopt($handle, CURLOPT_RETURNTRANSFER, true);
		
		curl_setopt($handle, CURLOPT_SSL_VERIFYPEER, false);
		
		curl_setopt($handle, CURLOPT_TIMEOUT, 20);
		
		$response = curl_exec($handle);
		
		curl_close($handle);
		
		return $response;
	}
	
	function bot_get_categories()
	{
		$response = bot_curl_get_content(BOT_BASE_URL);
		
		$list = preg_match(
			"#\<h(1|2|3|4|5|6)\>Kategoriler\<\/h(1|2|3|4|5|6)\>.*?\<ul\>(.*?)\<\/ul\>#si",
			$response,
			$matches
		) ? end($matches) : null;
		
		$items = preg_match_all("#href\=\"(.*?)\".*?\>(.*?)\<\/a\>#si", $list, $matches) ? $matches : null;
		
		$response = array();
		
		foreach(end($items) as $key => $value)
		{
			$response[] = (object) array("url" => $items[1][$key], "title" => html_entity_decode($items[2][$key], ENT_QUOTES, "UTF-8"));
		}
		
		return $response;
	}
	
	function bot_build_category_url($category, $page_number)
	{
		return $page_number <= 1 ? $category : rtrim($category, "/") . "/page/" . $page_number . "/";
	}
	
	function bot_get_category_posts($category, $page_number)
	{
		$url = bot_build_category_url($category, $page_number);
		
		$response = bot_curl_get_content($url);
		
		$items = preg_match_all("#class\=\"post\".*?\<h2\>\<a.*?href\=\"(.*?)\".*?\>(.*?)\<\/a\>#si", $response, $matches) ? $matches : null;
		
		$response = array();
		
		foreach(end($items) as $key => $value)
		{
			$response[] = (object) array("url" => $items[1][$key], "title" => html_entity_decode($items[2][$key], ENT_QUOTES, "UTF-8"));
		}
		
		return $response;
	}
	
	function bot_post_details($post_urls)
	{
		$response = array();
		
		foreach($post_urls as $post_url)
		{
			$post_response = bot_curl_get_content($post_url);
			
			$post_title = preg_match(
				"#name\=\"twitter\:title\".*?content\=\"(.*?)\"#si",
				$post_response,
				$matches
			) ? end($matches) : null;
			
			$post_description = preg_match(
				"#name\=\"twitter\:description\".*?content\=\"(.*?)\"#si",
				$post_response,
				$matches
			) ? end($matches) : null;
			
			$post_keywords = preg_match(
				"#name\=\"keywords\".*?content\=\"(.*?)\"#si",
				$post_response,
				$matches
			) ? implode(", ", array_map("trim", explode(",", end($matches)))) : null;
			
			$response[] = (object) array(
				"title" => html_entity_decode($post_title, ENT_QUOTES, "UTF-8"),
				"description" => html_entity_decode($post_description, ENT_QUOTES, "UTF-8"),
				"keywords" => html_entity_decode($post_keywords, ENT_QUOTES, "UTF-8")
			);
		}
		
		return $response;
	}
	
?><!DOCTYPE html>
<html class="no-js">
	<head>
		<meta charset="UTF-8" />
		<meta name="author" content="Ogün Karakuş" />
		<title>@imsaintx</title>
		<link rel="author" type="text/plain" href="http://saintx.net/humans.txt" />
		<link rel="icon" type="image/x-icon" href="http://saintx.net/storage/assets/images/favicon.ico" />
		<link rel="shortcut icon" type="image/x-icon" href="http://saintx.net/storage/assets/images/favicon.ico" />
		<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/modernizr/2.7.1/modernizr.min.js"></script>
		<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/jquery/1.11.0/jquery.min.js"></script>
		<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.1.0/js/bootstrap.min.js"></script>
		<script type="text/javascript">
			$(function() {
				Modernizr.load([
					"https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.1.0/css/bootstrap.min.css",
					"https://cdnjs.cloudflare.com/ajax/libs/font-awesome/3.2.1/css/font-awesome.min.css"
				]);
				
				$("body").css("padding", "1em");
				
				$("label, select, input[type=\"checkbox\"]").css("cursor", "pointer");
				
				$("h1.panel-title a").attr("target", "_blank").attr("title", document.title).append(document.title);
				
				$("span").tooltip();
				
				$("a").tooltip({placement: "right"});
				
				$(".full-width").css("width", "100%");
			});
		</script>
	</head>
	<body>
		<form method="post" class="panel panel-default form-horizontal">
			<div class="panel-heading">
				<h1 class="panel-title">
					<i class="icon icon-globe"></i>&nbsp;<a href="http://saintx.net/"></a>
				</h1>
			</div><?php if(empty($_POST)): echo PHP_EOL; ?>
			<div class="panel-body">
				<div class="form-group">
					<label for="bot_category" class="col-sm-1 control-label">
						<span class="label label-default">Kategoriler:</span>
					</label>
					<div class="col-sm-11">
						<select id="bot_category" name="bot_category" class="form-control">
							<?php $values = ""; foreach(bot_get_categories() as $category): $values .= sprintf("<option value=\"%s\">%s</option>\n\t\t\t\t\t\t\t", $category->url, $category->title); endforeach; echo trim($values) . PHP_EOL; ?>
						</select>
					</div>
				</div>
				<div class="form-group">
					<label for="bot_page_number" class="col-sm-1 control-label">
						<span class="label label-default" data-toggle="tooltip" title="Sayfa Numarası">Sayfa:</span>
					</label>
					<div class="col-sm-11">
						<select id="bot_page_number" name="bot_page_number" class="form-control">
							<?php $values = ""; foreach(range(1, 250) as $page_number): $values .= sprintf("<option value=\"%s\">%s</option>\n\t\t\t\t\t\t\t", $page_number, $page_number); endforeach; echo trim($values) . PHP_EOL; ?>
						</select>
					</div>
				</div>
			</div>
			<div class="panel-footer">
				<button type="submit" class="btn btn-default full-width">Öğeleri Getir</button>
			</div>
		</form>
<?php elseif(isset($_POST["bot_category"]) && isset($_POST["bot_page_number"])): echo PHP_EOL; $posts = bot_get_category_posts($_POST["bot_category"], $_POST["bot_page_number"]); ?>
			<div class="panel-body">
				<?php foreach($posts as $post): ?><div class="form-group">
					<label class="col-sm-1 control-label">
						<input type="checkbox" name="bot_posts[]" value="<?=$post->url;?>" />
					</label>
					<div class="col-sm-11">
						<span class="form-control">
							<i class="icon icon-external-link"></i>
							<a href="<?=$post->url;?>" target="_blank" data-toggle="tooltip" title="<?=$post->title;?>"><?=$post->title;?></a>
						</span>
					</div>
				</div>
			<?php endforeach; ?></div>
			<div class="panel-footer">
				<button type="submit" class="btn btn-default full-width">Öğeleri Getir</button>
			</div>
		</form>
<?php elseif(isset($_POST["bot_posts"])): echo PHP_EOL; $details = bot_post_details($_POST["bot_posts"]); ?>
			<div class="panel-body">
				<?php $last_key = count($details) - 1; foreach($details as $key => $post): ?><div class="form-group">
					<label for="post_title_<?=$key;?>" class="col-sm-1 control-label">
						<span class="label label-default">Yazı Başlığı:</span>
					</label>
					<div class="col-sm-11">
						<input type="text" id="post_title_<?=$key;?>" class="form-control" name="post_title" value="<?=$post->title;?>" />
					</div>
				</div>
				<div class="form-group">
					<label for="post_description_<?=$key;?>" class="col-sm-1 control-label">
						<span class="label label-default">Yazı Açıklaması:</span>
					</label>
					<div class="col-sm-11">
						<textarea id="post_description_<?=$key;?>" class="form-control" name="post_description" rows="5"><?=$post->description;?></textarea>
					</div>
				</div>
				<div class="form-group">
					<label for="post_keywords_<?=$key;?>" class="col-sm-1 control-label">
						<span class="label label-default">Yazı Etiketleri:</span>
					</label>
					<div class="col-sm-11">
						<input type="text" id="post_keywords_<?=$key;?>" class="form-control" name="post_keywords" value="<?=$post->keywords;?>" />
					</div>
				</div>
			<?php if($key != $last_key): echo "<hr />"; endif; endforeach; ?></div>
			<div class="panel-footer">
				<button type="submit" class="btn btn-default full-width">Öğeleri Getir</button>
			</div>
		</form>
<?php endif; ?>
		<footer class="text-center">
			<small>Copyright &copy; 2012 - <?=date("Y");?>, <a href="http://saintx.net/" target="_blank">Ogün Karakuş</a></small>
		</footer>
	</body>
</html>

@pastebin

ilgilendiğiniz için çok teşekkür ederim fakat siz yukarıda bayağı derine inmişsiniz ve neyin ne olduğunu yazmamışsınız dediğim gibi henüz öğrenme aşamasındayım kısacası hiç bişey anlamadım

11-02-2014, 13:36:35

#5

saintx

@hafsel;

Dökümante etmek hiç aklıma gelmedi.
11-02-2014, 13:39:44

#6

Vesgatex

Kimlik doğrulama veya yönetimden onay bekliyor.
hafsel adlı üyeden alıntı: mesajı görüntüle

merhaba...
Php ile bot yapmayı öğreniyorum takıldığım bir kısım var bu konuda bilgili arkadaşlardan yardım istiyorum..Şimdi derdimi örnek vererek anlatayım site.com/sayfa-1 diye bir sayfa var ben bundaki içerikleri alıyorum istediğim gibi değiştirip yazdırıyorum burda sorun yok.ama bunu yalnızca bir sayfa için yapabiliyorum...yapmak istediğim şey bot ile karşı siteden devam eden sayfalarıda almak..sayfa-1, sayfa-2 ,sayfa-3 .... Sayfa-x gibi otomatik devam edip içerik almasını istiyorum... Yardımlarınızı bekliyorum

for döngüsü kullanarak yapabilirsin.
```
for($i = 1; $i<= 10; $i++){ //10 sayfa sayısı
$site = file_get_contents("http://site.com/sayfa-".$i);
/* her bir sayfa için yapılacak işlemler */
}
```
11-02-2014, 14:12:19

#7

~~hafsel~~

Üyeliği durduruldu

evet for döngüsü ile bir adım daha ilerledim hepnize teşeklür ederim...