Tuesday, December 06, 2016

How to set the MAC address when connecting to wireless using /etc/network/interfaces

If you try to set the mac address using hwaddress ether xx:xx:xx:xx:xx:xx in /etc/network/interfaces, that fails. It seems ifup tries to set the address after connecting to WiFi. The address needs to be set before, while the interface is down. If you need the MAC address to connect, you won't connect, and if you don't need it and connect, you'll get RTNETLINK answers: Device or resource busy at the end. The solution is to use a pre-up ifconfig wlan0 hw ether xx:xx:xx:xx:xx:xx line instead of the hwaddress ether line.

Monday, December 05, 2016

Exiftran for Windows

The Windoze 10 Photos app doesn’t actually rotate JPEG files, and instead just sets the EXIF orientation. I want my photos to always be displayed in the correct orientation, so this is not acceptable, because the flag is ignored in many situations. Photos can be rotated with jpegtran, but my camera photos also contain a JPEG thumbnail in the Exif data, which also needs to be rotated. I used exiftran for this in Linux and I cross-compiled it for Win32 from Linux using i686-w64-mingw32-gcc. Download the build directory here. I'm distributing the whole thing because of the GPL. To use exiftran, rename exiftran to exiftran.exe and run it. For example, to rotate a file by 90 degrees in place, replacing it with a losslessly rotated version, use "exiftran -9i file".

Monday, November 28, 2016

Switching from IDE to AHCI on an ICH7-M based laptop

Many older laptops have SATA controllers and SATA drives, but set up the chipset in IDE mode instead of AHCI mode. Compared to IDE mode, AHCI can provide faster transfer rates, NCQ and power savings. It may not be possible to select AHCI mode in the BIOS, but it can be done by writing to PCI configuration registers directly. The actual IDE (PATA) ports are not supported in AHCI mode, so a PATA DVD drive would not be usable when AHCI is enabled.

You can read and write PCI registers using setpci in Linux, but if you want to make this change, it must be done before the device is detected. One way to do it is via a kernel patch, but recompiling the kernel every time for updates is kind of annoying. Another way is via GRUB's setpci command. This is very easy, and works well, but breaks resume from sleep, because GRUB doesn't run then and the kernel will encounter the device in IDE mode.

How to do this via GRUB

First, you need to find your controller. Use lspci and look at the output. On my Inspiron 6400 I find "00:1f.2 IDE interface: Intel Corporation 82801GBM/GHM (ICH7-M Family) SATA Controller [IDE mode] (rev 01)". The 00:1f.2 is the device's identifier in Linux, and lspci -n | grep 00:1f.2 shows "00:1f.2 0101: 8086:27c4 (rev 01)" This gives you the PCI ID of the device: 8086:27c4.

According to the ICH-7 datasheet PDF, you need to set the 8-bit MAP—Address Map Register at offset 0x90 to 0x40. (A patch I had before also set the SCRAE bit in the register at offset 0x94, but that just provides access to some AHCI registers when AHCI is not enabled, and is irrelevant when AHCI is enabled.) This means the following GRUB command is needed.

setpci -d 8086:27c4 90.b=40

You can press e when the GRUB menu is displayed, and add it to the end of the command that boots Linux. Adding it earlier caused a hang, probably because the BIOS cannot handle AHCI mode, and so it needs to be set after the kernel and initrd are loaded. After finding this works, I added an /etc/grub.d/42_ahci file with a stripped down Linux menu entry taken from /boot/grub/grub.cfg. It would be better if I could call the Ubuntu menu entry as a subroutine, but that doesn't seem possible with GRUB.

#!/bin/sh
exec tail -n +3 $0
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
menuentry 'Ubuntu AHCI' --class ubuntu --class gnu-linux --class gnu --class os {
recordfail
insmod ext2
set root='hd0,msdosPARTITION_NUMBER_HERE'
linux /vmlinuz root=UUID=PUT_UUID_HERE ro resume=SWAP_PARTITION
initrd /initrd.img
setpci -d 8086:27c4 90.b=40
}

The Results

This change increases sudo nice -n -19 dd if=/dev/sda bs=1M of=/dev/null count=1000 speed from 128 MB/s to 141 MB/s with a PNY SSD2SC120G1CS1754D117-551 SSD. It also enables NCQ, and causes the DVD drive to not be detected at all. Here are the IDE lines which disappear from dmesg. Note that this is based on diff output and is just a list of lines in order, not a continuous section.

pci 0000:00:1f.2: [8086:27c4] type 00 class 0x010180
pci 0000:00:1f.2: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7]
pci 0000:00:1f.2: legacy IDE quirk: reg 0x14: [io 0x03f6]
pci 0000:00:1f.2: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177]
pci 0000:00:1f.2: legacy IDE quirk: reg 0x1c: [io 0x0376]
ata_piix 0000:00:1f.2: version 2.13
ata_piix 0000:00:1f.2: MAP [ P0 P2 IDE IDE ]
scsi host0: ata_piix
scsi host1: ata_piix
ata1: SATA max UDMA/133 cmd 0x1f0 ctl 0x3f6 bmdma 0xbfa0 irq 14
ata2: PATA max UDMA/100 cmd 0x170 ctl 0x376 bmdma 0xbfa8 irq 15
ata2.00: ATAPI: SONY DVD+/-RW DW-Q58A, UDS2, max UDMA/33
ata1.00: 234441648 sectors, multi 2: LBA48 NCQ (depth 0/32)
ata2.00: configured for UDMA/33
scsi 1:0:0:0: CD-ROM SONY DVD+-RW DW-Q58A UDS2 PQ: 0 ANSI: 5
sr 1:0:0:0: [sr0] scsi3-mmc drive: 24x/24x writer cd/rw xa/form2 cdda tray
cdrom: Uniform CD-ROM driver Revision: 3.20
sr 1:0:0:0: Attached scsi CD-ROM sr0
sr 1:0:0:0: Attached scsi generic sg1 type 5


Here are the new lines which appear instead:

pci 0000:00:1f.2: [8086:27c5] type 00 class 0x010601
pci 0000:00:1f.2: reg 0x24: [mem 0x00000000-0x000003ff]
pci 0000:00:1f.2: BAR 5: assigned [mem 0x80000000-0x800003ff]
ahci 0000:00:1f.2: version 3.0
ahci 0000:00:1f.2: enabling device (0005 -> 0007)
ahci 0000:00:1f.2: forcing PORTS_IMPL to 0xf
ahci 0000:00:1f.2: SSS flag set, parallel bus scan disabled
ahci 0000:00:1f.2: AHCI 0001.0100 32 slots 4 ports 1.5 Gbps 0xf impl SATA mode
ahci 0000:00:1f.2: flags: 64bit ncq ilck stag pm led clo pmp pio slum part
scsi host0: ahci
scsi host1: ahci
scsi host2: ahci
scsi host3: ahci
ata1: SATA max UDMA/133 abar m1024@0x80000000 port 0x80000100 irq 27
ata2: SATA max UDMA/133 abar m1024@0x80000000 port 0x80000180 irq 27
ata3: SATA max UDMA/133 abar m1024@0x80000000 port 0x80000200 irq 27
ata4: SATA max UDMA/133 abar m1024@0x80000000 port 0x80000280 irq 27
ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
ata1.00: 234441648 sectors, multi 2: LBA48 NCQ (depth 31/32), AA
ata2: failed to resume link (SControl 0)
ata2: SATA link down (SStatus 0 SControl 0)
ata3: SATA link down (SStatus 0 SControl 300)
ata4: failed to resume link (SControl 0)
ata4: SATA link down (SStatus 0 SControl 0)

A better solution

Another alternative, which can support suspend to RAM, is to set registers from an altered DSDT. Read about the DSDT changes here and read about how to use a custom DSDT in Linux here. You first need to dump your DSDT, decompile it and fix it so it can compile properly. People do this sort of thing for running Mac OS on a PC, and you can find plenty of info on sites relating to that. Then you can add your modifications. In Ubuntu, simply copy the compiled dsdt.aml to /boot and run update-grub.

Tuesday, November 22, 2016

Sharing an IR receiver via an opto-isolator

I wanted to share an infrared remote receiver between the Pace DC550D cable box and Raspberry Pi 2 B running Kodi. This way there is no need to mount another visible IR receiver. It turned out to be a harder than anticipated because I used the wrong opto-isolator.

IR Receivers

An IR receiver has 3 connections: ground, power and signal output. With many IR receivers, the output is open collector and active low. This means when there is no remote signal detected, the output is not driven, and when a signal is detected, it is driven low. In order to actually see a voltage change at the output, a pull-up resistor is needed. Then the output stays high with no signal and goes low when a signal is detected.

The Pace DC550D, like many devices, connects to the IR receiver via a standard headphone jack. The tip supplies 5 V power, the ring (middle contact) is the signal output, and the sleeve is ground. This arrangement makes the most sense, but I don't know if every other device uses it.

The signal stays at just over 3 V normally and goes low when IR is detected. This might be because it connects to 3.3 V device. With an open collector output, the pull-up resistor does not need to connect to the same voltage as the device's power supply. So, an IR receiver powered by 5 V and with the pull-up resistor connected to 3.3 V will properly interface with 3.3 V devices.

Opto-isolation

Accessing the IR receiver signal is easy via headphone splitter and headphone plug. It ought to be possible to connect the IR receiver directly to the Raspberry Pi. Everything is grounded together via HDMI, though also connecting grounds would be good. However, I wanted opto-isolation for protection.

I connected the opto-isolator LED between 5 V power and signal output. This meant the LED would come on when a signal is detected. This seems better because it is drawing power through the IR receiver output, not the pull-up resistor. Though there is a problem because the off state is just over 3 V and LED voltage drop is only 1 V. I added 3 diodes in series to deal with this. A current limiting resistor is also required for the LED.

At first I tried to just connect the opto-isolator output to ground and a GPIO pin, and use the Raspberry Pi's internal pull-up. First, this did not work at all, because the lirc_rpi module parameter for setting that is ignored. Pull up can either be enabled by running a short Python program every time, or by by adding a dtparam=gpio_in_pull=up line after the dtoverlay=lirc-rpi line in /boot/config.txt. With the pull-up enabled, I got a signal, but it was useless.

The problem was the turn-off time of the optocoupler. I chose a TIL113 with photodarlington output, so that I would need to draw less power from the IR receiver. However, photodarlington opto-isloators have very slow shutoff. Adding a base resistor didn't help much, and even with a pull-up to 3.3 V on the output, LIRC was unreliable. Eventually I changed to a PS2010 with a just a normal phototransistor. Due to its much lower current transfer ratio, I had to carefully choose LED current limiting and output pull-up resistors. The output pull-up was still necessary; the Raspberry Pi's internal pull-up did not give a good enough signal.

The Results

This seems fully reliable now. I'm using a Harmony 300 remote, with the device set to Microsoft Media Player [Microsoft Xbox Media Center Extender]. It maps to the mceusb LIRC remote. The MyHarmony software doesn't configure all the possible keys, so it is necessary to edit the key configuration. The Prev button, corresponding to KEY_BACK is especially important for the Kodi interface.

Monday, November 21, 2016

Connecting to WiFi using Debian's /etc/network/interfaces

It's fairly easy to connect to a wireless network from the command line using /etc/network/interfaces if you know what you're doing. When connecting to a network with WPA or WPA2 encryption, you will need wpa_supplicant, but you don't need to create a conf file. Instead, options corresponding to wpa_supplicant.conf options can be placed directly in /etc/network/interfaces. Those options aren't documented in the man page, but you can find them in /usr/share/doc/wpasupplicant/examples/wpa_supplicant.conf.gz . Add a wpa- prefix and change the underscore to a dash. For example, scan_ssid in wpa_supplicant.conf becomes wpa-scan-ssid in /etc/network/interfaces. Something like this was suggested elsewhere.

iface wlan0 inet dhcp
    wpa-scan-ssid 1
    wpa-ap-scan 1
    wpa-key-mgmt WPA-PSK
    wpa-proto RSN WPA
    wpa-pairwise CCMP TKIP
    wpa-group CCMP TKIP
    wpa-ssid "<your ssid>"
    wpa-psk "<your password>

You only need wpa-scan-ssid 1 if you want to connect to a non-broadcasting network. Some of the other parameters have sensible defaults. Note that RSN is WPA2 and CCMP is the most secure encryption. The wpa-ssid quotes will be stripped, so you can use a SSID with leading and/or trailing spaces. This is not possible with wireless-essid.

You can also use other /etc/network/interfaces options, like an allow-hotplug wlan0 line so the interface gets configured when you plug in the dongle, or a static IP instead of DHCP. The interfaces(5) man page only lists a few of the options. Others are provided by shell scripts installed by other packages, like /etc/wpa_supplicant/functions.sh for the wpa- options.

Once everything is done, you can use sudo ifup wlan0 to connect. For some reason, that command can become unresponsive to attempts to kill it, and the ifup process may need to be killed elsewhere. Use sudo ifdown wlan0 to disconnect.

Thursday, November 17, 2016

Using Pidgin and purple-facebook for Facebook chat may also cause problems. App passwords may help.

Earlier I had problems with the Miranda NG Facebook plugin causing captchas on Facebook. I would get captchas even when posting and sharing very obviously innocent stuff. The plugin would get updated, and captchas would stop, but then they would start again later. So, I gave up on that and started using Pidgin with purple-facebook in Windows as well.

I had been using Pidgin with Purple Facebook in Linux for a while, but when I started using it in Windows, Facebook reacted as if my account had been hacked. I was forced to change my password, and examine recent activity. This problem repeated. I don't know if it's related to use of purple-facebook in Windows or use of it in too many places or a newer version or what. I reported the bug, and other users responded that they also encountered the problem.

Facebook has an app passwords feature which allows you to auto-generate passwords and use those passwords for apps instead of your main Facebook password. I'm now using that feature, with separate app passwords for every different Pidgin installation. So far so good.

Oh, and by the way, if Facebook forces you to change your password, don't bother trying to re-use the old password. They seem to blacklist the passwords used when their system thinks your account was hacked.

How to obtain higher resolution photos by altering the image URL

Often websites scale down the photos which they display. If you carefully examine the image URL and change it, you may be able to increase the resolution and quality. Look for parameters which specify the size and try to change them or strip them away. Also, if the photo URL refers to another path, try to directly access that other path.

You can often get an image URL by right clicking on the image. If that does not work, use your browser's developer tools or the media tab of page info in Firefox. The inspect feature of developer tools should make finding the image fairly easy.

Here are a few examples of these transformations:

http://wpmedia.windsorstar.com/2016/11/1d1.jpg?quality=55&strip=all&w=840&h=630&crop=1
http://wpmedia.windsorstar.com/2016/11/1d1.jpg

This was pretty obvious. Remove the parameters and get the original image size. It also seems to avoid JPEG re-encoding, and the accompanying increase in artifacts.

http://www.amherstburg.ca/ThumbGen.ashx?%2fAreas%2fCustom%2fContentFiles%2fportrait+of+ferry+and+shoreline.png/600/452/w
http://www.amherstburg.ca/Areas/Custom/ContentFiles/portrait%20of%20ferry%20and%20shoreline.png

In this case, you see another path given as a parameter to ThumbGen.ashx. You need to remove the encoding of that path. Note that the path is encoded with percent encoding and + instead of spaces.

http://i2.cdn.cnn.com/cnnnext/dam/assets/161117110259-dutch-war-ship-exlarge-169.jpg
http://i2.cdn.cnn.com/cnnnext/dam/assets/161117110259-dutch-war-ship.jpg

In this case part of the "file name" was the culprit, and the change removes cropping and text.

https://images-na.ssl-images-amazon.com/images/I/51dIUaKqkYL._SY500_.jpg
https://images-na.ssl-images-amazon.com/images/I/51dIUaKqkYL.jpg

Again, the last part of the file name before the .jpg had to be removed.

https://c1.staticflickr.com/6/5722/22981183464_8c7035afbb_b.jpg
https://c1.staticflickr.com/6/5722/22981183464_8c7035afbb_h.jpg

For some sites, there are site-specific tricks which are not self-explanatory. On Flickr, try changing the last letter after the underscore to h. In the past o was needed for some photos and may still be needed sometimes.

http://s7d5.scene7.com/is/image/CanadianTire/0431755_1?defaultImage=image_na_EN&wid=160&hei=160&op_sharpen=1
http://s7d5.scene7.com/is/image/CanadianTire/0431755_1?defaultImage=image_na_EN&wid=1600&hei=1600&op_sharpen=1

In this case, removing the parameters returns the default small size, but you can change the parameters to increase image size. Note that it would be almost entirely pointless to increase the size beyond the original size. You would just get a larger file without added detail. On some sites, if you specify a size that's larger than the original size, you will get the original size.

If you are not satisfied with what you can obtain at one site, then try Google search by image or TinEye. Remember that even if the search engine doesn't find bigger images, another site may allow you to download a larger image by modifying the URL.

Thursday, September 01, 2016

Dropbox is not a reasonable way to publicly share files anymore

I first used Drop.io to share files associated with blog posts. When Facebook bought it and shut it down, I switched to using Dropbox. At the time, many people liked Dropbox, and I found it worked well. It's nice to be able to upload a file for public sharing by simply copying it to my Dropbox folder.

Today I logged in to change my password because of the big Dropbox security breach, and found this:

This is ridiculous because of how little information it provides. They don't tell you the bandwidth limit, how much you used, what links used a lot of bandwidth or when access could be restored. They don't even really tell you whether it was due to bandwidth or some other kind of abuse. I'm actually only using 280 MB of space total, the public folder is 205 MB, and files are generally small. So, I guess either the bandwidth limit is extremely low, or some files had become very popular.

I'm not going to pay money to distribute files to others for free. Also, I'm especially not going to pay money to a company which treats its customers like this. So, now I should find some other service, move my files over, and go through my blog and change the links. Changing links is going to be a lot of work, and I don't feel that's worthwhile. So, maybe another day. Suggestions for what service to use are welcome.

By the way, I recently got an e-mail from Dropbox saying that HTML documents will stop rendering in the browser. I don't think I'm sharing any HTML documents, but if I am that means they would need to be downloaded and then viewed. That's another reason to not use Dropbox. They seem to be pushing Dropbox Paper really hard now, with all the e-mail I've been getting about that, and this may be an attempt to get people to use that for documents.