- Posts: 38
- Thank you received: 0
clean up log files
7 years 9 months ago #14098
by thatisfun
clean up log files was created by thatisfun
Our ETL is in Production for a while, it generate a lot of logs. Like several Millions of log files.
We found we cannot create system backup if we don't delete logs from monitor tab. it just frozen the system if we create Repository backup.
Here is the question:
1. How to create Repository backup, if we have a lot of logs in monitor?
2. Does ETL has any tool or feature to clean old logs?
Thanks
We found we cannot create system backup if we don't delete logs from monitor tab. it just frozen the system if we create Repository backup.
Here is the question:
1. How to create Repository backup, if we have a lot of logs in monitor?
2. Does ETL has any tool or feature to clean old logs?
Thanks
Please Log in or Create an account to join the conversation.
- Peter.Jonson
-
- Offline
- Platinum Member
-
7 years 9 months ago #14099
by Peter.Jonson
Peter Jonson
ETL Developer
Replied by Peter.Jonson on topic clean up log files
What is your repository type?
Peter Jonson
ETL Developer
Please Log in or Create an account to join the conversation.
7 years 9 months ago #14100
by thatisfun
Replied by thatisfun on topic clean up log files
MS SQL Server
Please Log in or Create an account to join the conversation.
- Peter.Jonson
-
- Offline
- Platinum Member
-
7 years 9 months ago - 7 years 9 months ago #14101
by Peter.Jonson
Peter Jonson
ETL Developer
Replied by Peter.Jonson on topic clean up log files
You can use MS SQL Server backups as alternative
If you want to clear logs you can delete it by runnig script
delete from QUEUE_HISTORY
delete from QUEUE_ACTIONS_HISTORY
or
truncate table QUEUE_HISTORY
truncate table QUEUE_ACTIONS_HISTORY
www.etl-tools.com/wiki/knowledgebase:repository_tables
If you want to clear logs you can delete it by runnig script
delete from QUEUE_HISTORY
delete from QUEUE_ACTIONS_HISTORY
or
truncate table QUEUE_HISTORY
truncate table QUEUE_ACTIONS_HISTORY
www.etl-tools.com/wiki/knowledgebase:repository_tables
Peter Jonson
ETL Developer
Last edit: 7 years 9 months ago by Peter.Jonson.
Please Log in or Create an account to join the conversation.
7 years 9 months ago #14102
by thatisfun
Replied by thatisfun on topic clean up log files
What about the several million log files under C:\ETL\logs folder?
What about we want to keep a backup of current logs for a while for trouble shooting?
Do we have the stop ETL Scheduler . before we do backup? running live database backup scares me.(it is usually for Disaster recovery )
Does ETL have best practice for log management? when do clean it. how to clean it. how to back it up,
Repository Backup looks like just Don't do the job, when logs are big.
Thanks
What about we want to keep a backup of current logs for a while for trouble shooting?
Do we have the stop ETL Scheduler . before we do backup? running live database backup scares me.(it is usually for Disaster recovery )
Does ETL have best practice for log management? when do clean it. how to clean it. how to back it up,
Repository Backup looks like just Don't do the job, when logs are big.
Thanks
Please Log in or Create an account to join the conversation.
7 years 9 months ago #14104
by John
Replied by John on topic clean up log files
If you have million of logs files you probably have a lot of data in QUEUE_HISTORY, QUEUE_ACTIONS_HISTORY tables
If you delete data from them your repository will become much smaller
Regarding the log file you can always move them into different folder using file operation action.
For example you can move files older than 10 days and compress them
If you delete data from them your repository will become much smaller
Regarding the log file you can always move them into different folder using file operation action.
For example you can move files older than 10 days and compress them
Please Log in or Create an account to join the conversation.