My last posting talked about digitally signing software for downloads. Unfortunately this is only half of the solution. I am sure that Microsoft and Norton are working in most people's best interests, but this does not help the way that CnW operates. With CnW, the idea is to give users the latest updates (and even bug fixes) as soon as possible. This is done by updating the main software package maybe once or twice a month. The changes are small, but incrementally ensure that the produce evolves, and responds to customer feed back.
Part of IE9.0 and Norton security is how long any download has been on the internet. This period is monitored to see if there has been any negative feedback about the product. Norton suggests there is a 1 week learning time, which would man that CnW software would be viewed as dubious for maybe 50% of it's time.
The next plan to try and resolve this issue is to move to the common pattern of having a installer tool, and then in effect an update program. In this case, the installer can be written, and will not be changed for a considerable period of time, maybe a year. The installer will then download the main, and frequently updated data from it's own secure environment. Everything will be digitally signed, but the installer can be recognised by IE9.0 and Norton as a stable, safe program. IE9.0 and Norton will only see the stable installer, though Norton will obviously also monitor any future updates.
Monday, July 4, 2011
Friday, July 1, 2011
Software signing and IE9.0
Internet security can be a large issue. To this end, IE9.0 has very 'tough' security measures added to try and protect users from downloading unsafe software. If a user tries to download an unsigned program, there are severe warning messages and it can be difficult to download or run the program.
To overcome the above problem the program has to be digitally signed, with a secure certificate. This can be an expensive process but fortunately there are solutions for small companies. One such company - that CnW have used - is Global Sign. This gives a digital signature which can then be traced back to Global Sign. Having this on the program now means that Internet Explorer V9 does not scream that the software could be very dangerous.
The other security measure that is causing problems is Norton Insight. This can warn that the software is new, and may be dangerous. The solution can be to get listed on their White List (is not a Black List). The problem here is that listing can take a few weeks, and with CnW this is when the next release may have been made. If a company does a single release each year this is not a problem, but for a small company doing regular updates to keep pace with new solutions, and customer requests, then Norton is not very friendly. Unfortunately, Norton does not seem to recognise the digital signature. I do not know what the ultimate solution will be.
To overcome the above problem the program has to be digitally signed, with a secure certificate. This can be an expensive process but fortunately there are solutions for small companies. One such company - that CnW have used - is Global Sign. This gives a digital signature which can then be traced back to Global Sign. Having this on the program now means that Internet Explorer V9 does not scream that the software could be very dangerous.
The other security measure that is causing problems is Norton Insight. This can warn that the software is new, and may be dangerous. The solution can be to get listed on their White List (is not a Black List). The problem here is that listing can take a few weeks, and with CnW this is when the next release may have been made. If a company does a single release each year this is not a problem, but for a small company doing regular updates to keep pace with new solutions, and customer requests, then Norton is not very friendly. Unfortunately, Norton does not seem to recognise the digital signature. I do not know what the ultimate solution will be.
Friday, June 3, 2011
Solid State devices
Many products are deemed to be dead long before they eventualy die entirely. Magnetic tape for datya storage is one, but there is a very important market for tape, though for the home user and small business, disk back up is now a much more viable proposition. The next prediction is the death of the rotating hard drive.
We are all used to solid state memory for cameras, telephones and many video devices. The capacity keeps increasing, and cost keeps coming down. Just starting to come in are solid state drives for laptops and desk top computers. As there is no head seek time, or rotational delay, reading can be very fast and many users are added them as the system drive in a PC. They report impressive performance in boot up and launching programs. It is noit all good news, as writing can be slow, and there is an issue of limited write cycles. Basically, sector will wear out if used too often. The solution to the last point is that chip controllers use a feature called wear leveling, so when a sector has been used too many times, it will be physically moved to another location, while still keeping the same logical address.
Data recovery of such SSDs has two main problems. If the controller dies, then it is necessary to work with the chips directly. These means they have to be unsoldered from the board and read. This means pointing a hot air gun at the chips and removing them, with out damage or over heating. It is possible, and not quite as bad as it sounds. The major problem though comes next. Manufacturers do not publish their wear leveling routines. As the chips are not meant to moved between devices, there is no requirement for any standards - all that matters is that when a sector number is requested, the correct data is returned. The physical sector is not relevant in any way.
CnW are now looking at such drives, and memory chips and will be developing tools to assist with recovery
We are all used to solid state memory for cameras, telephones and many video devices. The capacity keeps increasing, and cost keeps coming down. Just starting to come in are solid state drives for laptops and desk top computers. As there is no head seek time, or rotational delay, reading can be very fast and many users are added them as the system drive in a PC. They report impressive performance in boot up and launching programs. It is noit all good news, as writing can be slow, and there is an issue of limited write cycles. Basically, sector will wear out if used too often. The solution to the last point is that chip controllers use a feature called wear leveling, so when a sector has been used too many times, it will be physically moved to another location, while still keeping the same logical address.
Data recovery of such SSDs has two main problems. If the controller dies, then it is necessary to work with the chips directly. These means they have to be unsoldered from the board and read. This means pointing a hot air gun at the chips and removing them, with out damage or over heating. It is possible, and not quite as bad as it sounds. The major problem though comes next. Manufacturers do not publish their wear leveling routines. As the chips are not meant to moved between devices, there is no requirement for any standards - all that matters is that when a sector number is requested, the correct data is returned. The physical sector is not relevant in any way.
CnW are now looking at such drives, and memory chips and will be developing tools to assist with recovery
Thursday, May 26, 2011
Backup procedures
I often meet customers who say - I was about to back it up but....
Last week I suffered a serious problem with a Window 7 64 PC. It had become corrupted - probably due to device drivers - in a way that it would not respond to either mouse or keyboard. It booted up, I could see it over the network, but not control anything.
It is a Dell computer, and did not come with a Windows disk, but I had made a repair disk. I managed to boot into repair mode and first tried to go back a few restore points, however nothing worked.
I have a few backup prodcures in place, so was not too worried. The methods I have are
The first stage was to use the Microsoft image which is created evry week. The recovery mode allows for this to be restored, but it did not want to recognise the RAID box. However, the files were copied onto a USB drive, and was then visible. In the mean time, I took a complete image copy of the problem drive so that all updated files could be recovered as required.
The disk was then updaed with the microsft image, but this would not produce a bootable drive. It always came up in recovery mode, and often indicated that there were bad directories etc.
The final stage was to restore the partition, and boot sector from the Macrium backup. This was a few months old, but gave the pomise of a working system. I was very pleased that this worked, and it immediately booted and started running. The system then spent some time updating a few months of Norton and Microsoft patches. The recent file were copied back, some from disk, and some from carbonite, and all is now working correctly. No files or data lost.
My concern remains on how good the Microsoft backup system is. In the next few months I will try and receate a complete backup and see if it works.
Last week I suffered a serious problem with a Window 7 64 PC. It had become corrupted - probably due to device drivers - in a way that it would not respond to either mouse or keyboard. It booted up, I could see it over the network, but not control anything.
It is a Dell computer, and did not come with a Windows disk, but I had made a repair disk. I managed to boot into repair mode and first tried to go back a few restore points, however nothing worked.
I have a few backup prodcures in place, so was not too worried. The methods I have are
- On line carbonite
- Weekly Microsoft backup to a local RAID - in a separate box
- Periodic disk images using Macrium Reflect
- Very critical code (my source code) is backed up every 8 hours onto another PC, local but in a different room.
The first stage was to use the Microsoft image which is created evry week. The recovery mode allows for this to be restored, but it did not want to recognise the RAID box. However, the files were copied onto a USB drive, and was then visible. In the mean time, I took a complete image copy of the problem drive so that all updated files could be recovered as required.
The disk was then updaed with the microsft image, but this would not produce a bootable drive. It always came up in recovery mode, and often indicated that there were bad directories etc.
The final stage was to restore the partition, and boot sector from the Macrium backup. This was a few months old, but gave the pomise of a working system. I was very pleased that this worked, and it immediately booted and started running. The system then spent some time updating a few months of Norton and Microsoft patches. The recent file were copied back, some from disk, and some from carbonite, and all is now working correctly. No files or data lost.
My concern remains on how good the Microsoft backup system is. In the next few months I will try and receate a complete backup and see if it works.
Friday, May 6, 2011
DVD+RW apparently blank
I recently had a DVD+RW for recovery. It was from a camcorder and had probably been reformatted.
On initial examination the start of the disk was OK, but the majority was blank, ie all the sectors were filled with zeros. I could read the sectors and no error messages were displayed. Every indication was that the disk had been blanked, as if thee had been a full format.
When the disk was examined on hardware designed to read blank disks it was very interesting that the disk was not blank. There was a significant amount of video still on the disk and CnW software did a recovery, and then generated a new video disk.
The concern is that standard hardware gave every indication that the disk was blank - so do not rely on standardard hardware if there is a possibility that the data may actually still exist. CnW Recovery services will always assist anyone with such possible disks.
On initial examination the start of the disk was OK, but the majority was blank, ie all the sectors were filled with zeros. I could read the sectors and no error messages were displayed. Every indication was that the disk had been blanked, as if thee had been a full format.
When the disk was examined on hardware designed to read blank disks it was very interesting that the disk was not blank. There was a significant amount of video still on the disk and CnW software did a recovery, and then generated a new video disk.
The concern is that standard hardware gave every indication that the disk was blank - so do not rely on standardard hardware if there is a possibility that the data may actually still exist. CnW Recovery services will always assist anyone with such possible disks.
Tuesday, April 26, 2011
Finding owner of data
On a recent walk along a Californian beach I found a compact camera, covered in sand and salt water. The camera was very dead, but out of interest I eventally managed to extract the 2GB SD memory card. After drying it out, it was readable with errors. When totally dry, it read without errors.
I was keen to try and contact the owner and return the photos, over 500 of them. Unfortunately, there is very little on a memory chip to tell you about an owner, only the camera, and date and time of the photos. From this I could determine that the last photo had been taken about 10 days before I found the camera. I did try and add a 'Found camera' to a local lost and found website, but no reply.
Recently, I did a bit more investigation by looking at the photos more closely. Obviously the owner was a young person, with lots of photos in night clubs, no names identifiable, but also some at a college. One photo though I do hope will be a major clue is that it includes, possibly the owner, or close friend holding a college certificate with their unusal surname name on it. A bit of Googling took me to Facebook, and I hope a perfect match. (The name and college course both match, and the photo looks similar). I have now sent a message to the person, and am waiting for a reply.
The moral of the story is that if you want goods returned, it can be helpful to have some return details stored with it.
As a post script, the owner did contact me and a DVD of photos was sent. I have just received a very nice letter of thanks.
I was keen to try and contact the owner and return the photos, over 500 of them. Unfortunately, there is very little on a memory chip to tell you about an owner, only the camera, and date and time of the photos. From this I could determine that the last photo had been taken about 10 days before I found the camera. I did try and add a 'Found camera' to a local lost and found website, but no reply.
Recently, I did a bit more investigation by looking at the photos more closely. Obviously the owner was a young person, with lots of photos in night clubs, no names identifiable, but also some at a college. One photo though I do hope will be a major clue is that it includes, possibly the owner, or close friend holding a college certificate with their unusal surname name on it. A bit of Googling took me to Facebook, and I hope a perfect match. (The name and college course both match, and the photo looks similar). I have now sent a message to the person, and am waiting for a reply.
The moral of the story is that if you want goods returned, it can be helpful to have some return details stored with it.
As a post script, the owner did contact me and a DVD of photos was sent. I have just received a very nice letter of thanks.
Sunday, March 6, 2011
Virus issue
I don't often get involved in fixing PC problems, but recent jobs involved the same type of issue. The PC starts to report serious errors, and show problems with boot sector and other critical areas of the disk. It all looks rather real, and very worrying. However, in both cases it was caused by a similar, but different Fake Anti Virus program. I think if you continue there was an offer of the program to fix the issue - in exchange for some $$$s.
The approach I tooks was to remove the drive from the laptop and create an image for security purposes. I then ran Norton which tracked down several viruses, and removed them. This is where the two viruses behaved differently.
Virus one did not want to be removed by Norton and started each time the machine booted. The problem was that a startup function (go to msconfig) was launching the virus at startup each time. By removing this start up line - and seeing the program it was starting (it had a randomly generated name) the PC was then OK. The free AVG virus checker was added to the PC to try and prevent this happening again. A report a few weeks after this event indicated that eveything has been OK.
Virus two was removed by Norton, but left the PC in a state where no program would actually launch from explorer. Various 'Googled' ideas pointed to the registry, but this did not help. Launching the command prompt was also very difficult and the start program launcher did not work. A solution to this was rather unusual, but worked, and hence I am including it here. Do Ctrl-Alt_Del and b ring up the task manager. Under the top menu item 'File' thre is a run command. This worked and a command window was opened. It did not seem possible to change file attributes to make sure that a .exe was launched so evenually the PC was restored to a restore point from afew weeks earlier. Everything then nearly worked.
On examining the PC there were issue with McAfee antivirus which was not running, and also 18 months of Vista updates had not been loaded - 90 patches althogether. The 90 patches were installed, McAfee updated and this worked. PC now all OK, but all automatic updating was set to 4am when the PC is normally turned off. This was changed to a time when the PC was likely to be on and hopefully the problem will not arise again.
In both cases, no data was actually lost
The approach I tooks was to remove the drive from the laptop and create an image for security purposes. I then ran Norton which tracked down several viruses, and removed them. This is where the two viruses behaved differently.
Virus one did not want to be removed by Norton and started each time the machine booted. The problem was that a startup function (go to msconfig) was launching the virus at startup each time. By removing this start up line - and seeing the program it was starting (it had a randomly generated name) the PC was then OK. The free AVG virus checker was added to the PC to try and prevent this happening again. A report a few weeks after this event indicated that eveything has been OK.
Virus two was removed by Norton, but left the PC in a state where no program would actually launch from explorer. Various 'Googled' ideas pointed to the registry, but this did not help. Launching the command prompt was also very difficult and the start program launcher did not work. A solution to this was rather unusual, but worked, and hence I am including it here. Do Ctrl-Alt_Del and b ring up the task manager. Under the top menu item 'File' thre is a run command. This worked and a command window was opened. It did not seem possible to change file attributes to make sure that a .exe was launched so evenually the PC was restored to a restore point from afew weeks earlier. Everything then nearly worked.
On examining the PC there were issue with McAfee antivirus which was not running, and also 18 months of Vista updates had not been loaded - 90 patches althogether. The 90 patches were installed, McAfee updated and this worked. PC now all OK, but all automatic updating was set to 4am when the PC is normally turned off. This was changed to a time when the PC was likely to be on and hopefully the problem will not arise again.
In both cases, no data was actually lost
Thursday, March 3, 2011
Parallel Programming
Modern proessors are not getting much faster, 3GHz has been a fast processor for about 7 years. They are getting wider. It started with hyperthreading and then multiple cores. Almolst every processor is now a multiple core, and current Intel chips have 4 cores with hyperthreading, making in effect 8 cores.
Unfortunately, most programs just use a single core and so performance gains are not very significant. The solution is to use parallel programming so that different tasks are performed in different cores. This may sound simple but unfortunately many computing tasks are sequential. In data recovery it is sequence or read disk, analyse data and save data. The other problem is each time a task is split there is a processing overhead. This means that benefits may not be very significant.
An example of the limited benefit mentioned above is a simple program I wrote to experiment with parallel programming. It was purely an exercise with in memory manipulation - ie no hard disk access. The first example was single threaded and took 35 seconds to run, using a single core. The next example was using the 'parallel_invoke' function and used all possible cores. When running it looked impressive with all 8 cores running at 100%. However, the time was not reduced by a factor 8, but only roughly halved to 15 seconds. Although this would be a worth while time gain its shows how overheads of a new task eat into the gains. I am sure that a bit of tweaking could have made the improvement better, but the warning is that a PC may be running at 8 * 100% but actually alot of this may be house keeping.
In a real world example I have added some parallel processing into CnW Recovery software. The area was to do with calculating MD5 hash values while writing data to the output drive. As these processes do not depend on each other, they can run at the same time sharing the same memory buffer. The result was a reduction in time from about 3 hours 30 mins to 3 hours 10 mins. This is worth while but not very dramatic. However, it will be possible to add SHA-1 hashing with no extra time penalty and that would be a major benefit.
Unfortunately, most programs just use a single core and so performance gains are not very significant. The solution is to use parallel programming so that different tasks are performed in different cores. This may sound simple but unfortunately many computing tasks are sequential. In data recovery it is sequence or read disk, analyse data and save data. The other problem is each time a task is split there is a processing overhead. This means that benefits may not be very significant.
An example of the limited benefit mentioned above is a simple program I wrote to experiment with parallel programming. It was purely an exercise with in memory manipulation - ie no hard disk access. The first example was single threaded and took 35 seconds to run, using a single core. The next example was using the 'parallel_invoke' function and used all possible cores. When running it looked impressive with all 8 cores running at 100%. However, the time was not reduced by a factor 8, but only roughly halved to 15 seconds. Although this would be a worth while time gain its shows how overheads of a new task eat into the gains. I am sure that a bit of tweaking could have made the improvement better, but the warning is that a PC may be running at 8 * 100% but actually alot of this may be house keeping.
In a real world example I have added some parallel processing into CnW Recovery software. The area was to do with calculating MD5 hash values while writing data to the output drive. As these processes do not depend on each other, they can run at the same time sharing the same memory buffer. The result was a reduction in time from about 3 hours 30 mins to 3 hours 10 mins. This is worth while but not very dramatic. However, it will be possible to add SHA-1 hashing with no extra time penalty and that would be a major benefit.
Monday, February 7, 2011
WD disk not responding
A recnt Western Digital 500GB disk was spinning, but not recognised by a PC with USB connector. The next test I always try is with Ace Laboratories PC-3000 which is designed to test abd refirm driver firmware. At first this would not recognise the disk under general routines, but when the WD funtion tests were tried some response was found. It was not enough to recognise which family the drive belonged too, and so this was a case at looking at all the PCB layouts and matching a photo. Eventually the short list was 4 possible drive types, of which Tornado 2D was found to work.
The PC-3000 did a series of tests, and end of which it was possible to view a sector.
I did not know how long this reading would last so I started a Read to file and this moved very slowly, but positively. After a few hours the reading sped up to an acceptable rate. Over night over 200GB was imaged, but then the reading had gone painfully slow. I stopped the read and confirmed that other areas of the disk could be read without significant delays. The problem with PC-3000 is that it is not very good at reading sections of disks in an easily managed way.
The next stage was to carefully disconnect the SATA cable from PC-3000 and connect a PC (USB) SATA connector, without turning the drive power off. To my delight, the PC would read the disk anda process of incremental imaging was continued. I started with the final 200GB or so of the disk, and slowly worked back to the area that caused so many problems. When I had managed to image probably over 99% of the I decided the return rate was probably too small to consder worth continuing. The image produced a very healthy recovery of the disk, thanks both the PC-3000 and Incremental Imaging.
The PC-3000 did a series of tests, and end of which it was possible to view a sector.
I did not know how long this reading would last so I started a Read to file and this moved very slowly, but positively. After a few hours the reading sped up to an acceptable rate. Over night over 200GB was imaged, but then the reading had gone painfully slow. I stopped the read and confirmed that other areas of the disk could be read without significant delays. The problem with PC-3000 is that it is not very good at reading sections of disks in an easily managed way.
The next stage was to carefully disconnect the SATA cable from PC-3000 and connect a PC (USB) SATA connector, without turning the drive power off. To my delight, the PC would read the disk anda process of incremental imaging was continued. I started with the final 200GB or so of the disk, and slowly worked back to the area that caused so many problems. When I had managed to image probably over 99% of the I decided the return rate was probably too small to consder worth continuing. The image produced a very healthy recovery of the disk, thanks both the PC-3000 and Incremental Imaging.
Monday, January 31, 2011
Windows 7 update
I recently wrote about upgrading a PC from Vista to Windows 7. The cost was about £80 for software upgrade and £25 for extra memory.
It has been worth it. The machine is more reponsive and setting up network connections and printers much easier. The configuration is a 2.4GHz, Core 2 duo PC with 4GB RAM, 750GB hard drive, (250-500 partitions), a 2TB WD green drive, and a 2.7TB RAID. The only slow part to be investigated is the100Mb network.
The WD Green drives are not blistering fast, but they are very cool (in the sense of temperature). This should help reliablity, and mean that computer fans can run less often, giving me a quieter life.
Graphics are not important, so the PC has the standard graphics interface, but Windows 7 aero does work.
It has been worth it. The machine is more reponsive and setting up network connections and printers much easier. The configuration is a 2.4GHz, Core 2 duo PC with 4GB RAM, 750GB hard drive, (250-500 partitions), a 2TB WD green drive, and a 2.7TB RAID. The only slow part to be investigated is the100Mb network.
The WD Green drives are not blistering fast, but they are very cool (in the sense of temperature). This should help reliablity, and mean that computer fans can run less often, giving me a quieter life.
Graphics are not important, so the PC has the standard graphics interface, but Windows 7 aero does work.
Subscribe to:
Comments (Atom)