February 08, 2010
It has been quite some time since I last posted here about msec. For the past few weeks, it received some attention and now I guess many of the features I wanted to push for Mandriva 2010.1 are implemented. So I’ll describe the most interesting ones in this blog post (and save some for later 
).
First of all, starting with Mandriva 2010.1, msec will support user-defined periodicity for all periodic security checks. Therefore, it is possible to specify if each test should be executed daily (like in all previous msec versions), weekly or even monthly. In my opinion, this feature is one of the most interesting among all others, because it allows you to fine-tune the balance between security checks and daily I/O load caused by some expensive checks.
By default, checks which require lots of I/O (e.g., checking for unowned files, or world-writable files, and so on) will run weekly on the standard security level. Why so? Because this check was responsible for approximately 80% of all time required to run the periodic checks, and on most of the machines its results did not differ between consecutive days. Surely, it is nice to have a daily notification of all those changes, but the I/O cost of it is unacceptable high. Of course, you can define the periodicity of all such checks to be daily when you want, by using msecgui application of editing the configuration file manually .
Another interesting feature was the de-duplication of variables between main msec configuration file (security.conf) and the level configuration file (for example, level.standard). On previous versions, all variables were defined in security.conf, even if they have exactly the same value as the default one for the current security level. This way, it was easier to see all the configuration at once by looking into /etc/security/msec/security.conf file. On the other hand, it lead to duplication of almost all variables..
So for 2010.1, the behavior when saving the configuration file was modified to be more logical (and similar to the one of msecgui, which displays variables that differ from the default values for the security level in different way). If you want to redefine a variable, just specify it in security.conf and this change will take effect. If you want to disable a variable completely, just define it to an empty value (like, CHECK_SOMETHING=), like in previous versions, and it will be disabled.
To simplify this, we could use the following analogy: in previous msec versions (e.g., 2009.1 and 2010.0), the security.conf file contains the whole security configuration of msec plus the name of the security level which is used as base. In 2010.1, it contains the reference to the base security level plus only the variables which must be overridden for this level. In other words, on Mandriva 2010.1 just by looking at the msec security file it is possible to say “this machine is configured to use the same configuration as on standard security level, except those three checks that should be disabled).
There is yet another reason for this change, which will be described in details when it gets implemented (probably in a few coming weeks). So stay tuned for more news .
Another feature was the possibility of running the RedHat sectool checks periodically, among with all other msec checks. Just install sectool package from the contrib, and its checks will be executed automatically by msec.
Additionally, the integration between msec and msecperms applications was improved, making it easier to switch security levels and creating custom levels.
Besides those changes, several msec messages were improved to make them easier to understand by non-geek users , and, like usual, several bugs were fixed.
Like always, I am very interested in your feedback on those changed. Please, feel free to drop me a note whether you like these features, dislike them, or any other kind of comments about msec.
February 08, 2010 15:13
February 06, 2010
- Echo & The Bunnymen - The Killing Moon
- Bauhaus - Bela Lugosi's Dead
- Jesus & Mary Chain - Never Understand
- OMD - Maid of Orleans (The Waltz Joan of Arc)
- Siouxsie and the Banshees - Israel
- The Band of Holy Joy - Tactless
- The Bodines - Skanking Queens
- The Railway Children - A Gentle Sound
- U2 - New Years Day
- Gene Loves Jezebel - Motion Of Love
- Depeche Mode - A question Of Lust
- The Smiths - Barbarism Begins At Home
MixTape#1
February 06, 2010 03:56
February 05, 2010
O Bastter continua evoluindo o esquema dele, com ferramentas cada vez mais profissionais, mas sem perder a característica de pessoalidade. Dentro de um universo onde há quem prometa 30% ao mês de rendimento em mercados que são necessariamente soma-zero (o tal do Forex), o Bastter destaca-se como o mais honesto e didático dentre os consultores de investimentos.
Agora (ou há algum tempo, mas só vi agora), tem a "TV Bastter" (http://www.tvbastter.com), basicamente uma coleção de vídeos com dicas. (O único senão é o formato do vídeo, WMV; felizmente o Quicktime do Snow Leopard parece suportá-lo).
Dentre esses, há a série "Cemitério dos Malandros", com direito a entrada e locutor lembrando filmes do Zé do Caixão. Cada "filminho B" fala sobre uma armadilha em que investidores iniciantes, aqueles que querem viver de Bolsa (todo mundo já teve esse sonho, é fato) costumam cair.
Altamente recomendado, inclusive para rir um pouco e desestressar da quedona da Bovespa de ontem.
February 05, 2010 12:32
February 04, 2010
Segundo passo nos meus planos de dominação mundial, ops, construção de minha própria máquina de arcade: transplantar os componentes para um “gabinete” temporário para que eu possa deixá-la montada em um canto e facilitar os testes. Afinal de contas limpar a mesa da cozinha, pegar a placa, a fonte, o HD, ligar tudo, catar o monitor do Gabriel, etc e tal não é produtivo.
O plano: dar um pulo em uma loja de materiais para arte, comprar placas de MDF (um tipo de compensado, mais resistente) e montar um caixote para abrigar os componentes. Mas minha preguiça, combinada ao mau-tempo constante em São Paulo nos últimos meses, interferiu e resolvi não sair de casa.
Plano B: seguir o conselho do Leandro Pereira, que disse no Twitter: “monta dentro da caixa”. Faz sentido, a placa-mãe veio dentro de uma caixa de papelão razoavelmente resistente e do tamanho certinho. Não caberia a fonte, mas ela é bem protegida e pode ficar externa, nem o HD, mas para testes iniciais um pendrive de 16 GB dá e sobra. Então mãos à obra!
Umas duas horas, alguns furos e cortes e um pouco de cola quente depois nascia o Gambiarra I. Acho que quebrei TODAS as “boas-práticas” estabelecidas para montagem de PCs nessa máquina, mas ela funciona e cumpre seu propósito. A placa-mãe está parafusada a espaçadores de latão presos ao fundo da caixa. O espelho com os conectores é encaixado em um corte na traseira, e outro na frente dá espaço para um botão de força e um LED.
Liguei o micro na TV de 32″ da sala e instalei o Ubuntu em um pendrive para um teste rápido. Por enquanto, tudo OK. Só preciso melhorar a ventilação da “caixa”: o clock do processador é automaticamente reduzido com o aumento da temperatura, e vi Street Fighter Alpha 3 cair de 60 FPS sólidos (com a tampa aberta, por isso os pregadores nas fotos) para 43 com ela fechada. Nada que mais alguns cortes nos lugares certos não resolvam.
Por enquanto meu arcade é um PC numa caixa de papelão ligada à TV. O próximo passo é definir um sistema operacional e front-end, e iniciar o projeto dos controles. Espero ter essa parte pronta até o final das minhas férias, no início de março. Até lá, já posso ir jogando Street Fighter “na telona”. O problema é me acostumar com as 32 polegadas e voltar pra um CRT de 17 depois…
February 04, 2010 04:22
February 03, 2010
Then one day you say the wrong thing to the wrong person and bam!, you’re fired
and forgotten,
shaken from Maya without even a goodbye,
blown by the wind in the streets
with waves of other discarded bourgeois youth,
we the losers.
The Machine hungers for new blood.
A quarter century, and back to the starting point. Not much to show
for all the trouble — a couple babies, a résumé full of job-hopping, a few boxes of the antidepressants I now can’t afford, too many
books, too much debt. It doesn’t matter. In the town
nothing’s real except the town. It’s morning and grandma is
baking me cake.
February 03, 2010 22:38
February 01, 2010
⁂
I consider it a mistake to have discussed my clinical depression in this blog. The general public is basically ignorant about mental diseases, and with the safety of distance, people in the Internet have a belligerence they wouldn’t dream of showing in real life (the “Internet Tough Guy” phenomenon). If you have mental problems, that’s …not good.
Nonetheless it did have a few good effects. After I came out this particular closet, several people have privately contacted me about their own problems, telling me of how much my posts helped them. That alone makes it worth to have endured the trolls. When you’re mentally ill, it’s very important to find out you aren’t the only one.
Darryl Cunningham’s Psychiatric Tales was that important to me, and I always recommend it to my friends who need the same kind of help. However, I can’t seem to find an index, and I always struggle to find all chapters. This time I’m writing it down for later reference. Because it could be useful for someone, I’m posting the list.
I suspect there’s an index somewhere and I’m being a dummy, but who knows—one more cannot hurt.
February 01, 2010 23:19
January 30, 2010
This afternoon I put the last block into Python QAM "modem" version 3: Automatic Gain Control (AGC). The sources can be found at http://epx.com.br/wav, the m3*.py files, as well as gray.py as a dependency. The feature can be tested by e.g. changing volume of qam.wav file using Audacity.
AGC allows the modem to adapt to different power levels. Up to now, V3 modem was not adapting to the "volume" in WAV file; it assumed that TX always used 90% of dynamic range. V2 already had a crude amplitude measurer which only paid attention to the training header sequence. But a true AGC must adapt to changes mid-flight.
This is easier said than done. The main problem is that AGC must figure out the full dynamic range, that is, the maximum amplitude a symbol may have. But there is no guarantee that the "strongest" symbols will actually appear in QAM signal. And, if they appear, how does AGC "know" that they are the ones?
A crude solution would be to send the training sequence at maximum amplitude every few seconds, which is easily detectable as a no-data sound, but this wastes bandwidth and is not 1337 enough :)
The actual solution that real-world modems employ is to work with power level averages. It is trivial to calculate the average amplitude of our constellation. Then, we maintain a moving average of QAM signal reception, and compensate for the difference.
V3 uses a one-second moving average, which is short enough to profit from initial training sequence, and adapts fast enough while actual data is being received. TX was changed to send a training sequence at this average power level instead of the maximum possible power level, so AGC does not have to distinguish between training and data signal parts.
Then we have another problem: will actual signal power average match the theoretical constellation average? In theory, this will happen only if all symbols are equally probable. Real-world modems employ a randomizer on bit stream to ensure that.
I was optimistic that AGC would get a good average without a randomizer, but I was wrong. AGC will not work at all without a very random bit stream! It is incredible how ordinary data, like a text file, is so visibly non-random at such a low level.
So, out of necessity, I have implemented a very simple bit randomizer for V3, with a "4th order polynominal". This is a very pedantic way to say that randomizer takes into account the last 4 bits to figure out the next one, which is calculated with simple XOR operations. To be even more pedantic, I even employed a good programming practice this time: a self-test.
# Randomizer: 1 + x**-3 + x **-4
def randomize(b0):
global b1, b2, b3, b4
bs = b0 ^ (b3 ^ b4)
b1, b2, b3, b4 = bs, b1, b2, b3
return bs
def derandomize(b0):
global b1, b2, b3, b4
bt = b0 ^ (b3 ^ b4)
b1, b2, b3, b4 = b0, b1, b2, b3
return bt
rseq = [ int(random.random() * 2) for x in range(0, 25) ]
b1, b2, b3, b4 = (0, 0, 0, 0)
rseqS = [ randomize(b) for b in rseq ]
b1, b2, b3, b4 = (0, 0, 0, 0)
rseqT = [ derandomize(b) for b in rseqS ]
b1, b2, b3, b4 = (0, 0, 0, 0)
if rseqT != rseq:
print "Randomizer is broken"
print rseq
print rseqS
print rseqT
sys.exit(1)
This randomization scheme was copied directly from Fabio Montoro's "Modem e transmissão de dados" book.
It was funny to realize in practice how important the randomizer is, when you simply don't know the power level at RX side and must figure it out based on data stream itself.
Determining power level using averages, even with the help of randomizer, does not have the same precision as knowing it beforehand. This reduces the number of bits per symbol that modem can handle. V3 had reached 16 bits/symbol, now the highest reliable level is 12. (Turning off AGC in RX side makes it go back to 16.)
AGC was the last thing to implement in V3. The next major milestone, V4, will add some form of convolutional coding to increase noise resistance.
January 30, 2010 03:33
January 29, 2010
This is my reaction to The iPad provides the ultimate browsing experience? that appeared on the Flash Blog.
I’m not an Apple fan but I tend to disagree with that campaign of Flash support.
Clearly, this is another platform war. The Apple platform versus the Flash platform. It is very strategic for Apple to keep the attention of developers on the platforms and technologies they support. Supporting Flash means the opposite for Apple, so this is why they won’t allow it.
In this case I’ll be with Apple because their platform is the pure browser, standard, open HTML5 with XML-SVG and JavaScript – which all sum a platform as exciting, beautiful, colorful, dynamic, interactive as the proprietary Flash, but open.
Flash is proprietary, bad for SEO, bad for the semantic web, bad for deep linking, bad for the customer, good only for lazy web developers.
I hate visiting websites entirely built on Flash.
So if you, web developer, want to support your web apps on the iPad, iPhone, iTouch etc, just use standard web technologies (to cite them again: HTML5, SVG, JavaScript). Oh, by the way, all other browser will be (or should be) supported right away because these are the good technologies.
The only thing Flash is still useful for is playing streamed MP4+H.264 video. Nothing else.
© Avi for Avi Alkalay, 2010. |
Permalink |
No comment |
reactions to by Google and Technorati |
Add to
del.icio.us
Classified by author as: Multimedia, Web 2.0
Tagged by author with:
Feed enhanced by Better Feed from Ozh
January 29, 2010 14:29
The graph above is the waveform for the 38400 bps QAM signal. While I believe in math, my common sense refuses to believe that any information can be recovered from such a chaotic wave.
The "version 3" QAM modem written in Python is a real winner. Today, some of the FIXMEs in code were implemented, and the thing reached 38400 bps at 2400 baud, and 48000 bps at 3000 baud -- both using 16 bits per symbol. Note that 3000 baud is almost twice the rate of 1800Hz carrier!
It "wanted" to work at 18 bits/symbol, but showed too many errors. I am not sure whether 16-bit WAV quantization is the hard limit here, but it seems to be the case. I was quite surprised to see that transmitted message was still recognizable, albeit corrupted.
The changes in v3 which allowed this performance jump are:
1) added code to detect adjacent symbols which are equal in amplitude and phase. While all V3 constellation symbols cause a phase change, sometimes it is very small, especially if we are sending many bits per symbol.
The algorithm is simple: if a symbol change is not detected within the expected interval, the complex signal is sampled anyway. Idiot as it seems, it allows RX to handle up to 16 bits per symbol, while it handled only 7 before.
2) Added a carrier Hz compensation, PLL-like. Any difference between TX and RX carriers cause a "phase drift" in decoded QAM signal -- that is, phase shows a small but continuous rotation in absolutely every sample. This can be distinguished from sudden, time-limited symbol phase rotations, and compensated for.
I discovered that phase drift is more stable (and hence easier to handle) if RX carrier frequency is smaller than TX. If TX carrier is above TX, phase drift oscilates in a difficult way. So, now we make sure that RX carrier is always "low". For symbols with 16 bits each, RX can tolerate up to 15 Hz carrier deviation, which seems very good to me.
3) Constellation uses Gray code now, so adjacent points are guaranteed to have only one different bit. This "softens" errors (even if RX resolves to the wrong symbol, chances are just one bit will be corrupted), and helps whatever error-correction algorithms in use (currently we use none, but I plan to implement convolutional codes in V4).
Towards v4
Actually, the v3 performance numbers are getting meaningless; of course they are possible just because the medium is a noiseless 16-bit WAV file, whose Shannon capacity is almost 700 kbps. (The Shannon limit of a phone line is around 24kbps.)
The next challenge is to make RX work well on noisy and band-limited signals. Rates like 14400bps will be possible only by implementation of convolutional code, I guess.
January 29, 2010 05:23
January 28, 2010
Today I spent several hours testing the almost amazing Samsung LED TV on a local store. More specifically the UN40B7000 model.
The image quality is great and all but I was more interested in the digital multimedia features included in these new models, branded Medi@ 2.0 (or Media 2.0). You actually plug these TVs on your home network and can access media content on a home DLNA/UPnP server or the Internet.
Well, I tested almost all features and I can say it is a good product but its not ready for me to buy, and let me point why:
- MKV (Matroska) video files with DTS audio tracks won’t play audio. I got a message on the screen saying the audio codec is not recognized. Well, DTS is pretty popular.
- Chapter information embedded on MKV video files will be ignored. Also, if you want fast forward a movie, 2x is the fastest you can get.
- There is no way to choose which audio track embedded on a MKV or MP4 file will be played, in case you have video files with multiple audio tracks.
- Subtitles embedded in MKV and MP4 movie files are ignored. Apparently only external subtitles files are supported but I didn’t tested it because I embed all my videos with their subtitles. Anyway, external subtitle files may lead to other problems such as charset selection and I couldn’t find any menu option for such things.
- To search the handy embedded YouTube application you have to use the numeric keypad on the remote control as you type SMS in those old fashioned cell phones.
- I can live with all the above but there is one unacceptable super irritating limitation: you can’t browse other files or slideshow photos while music is being played. So you browse your MP3s, select an album to play and you are stuck there. If you try to go back to the menus, the music will stop playing. This is soooo 1980.
- It is very difficult to find complete specifications for this TV. For example, the list of supported codecs etc. Samsung website has a nice design but is very non practical and doesn’t provide enough information.
- A friend that owns this TV claimed the (very expensive) Samsung USB WiFi accessory for this TV can’t connect to his WPA secured wireless network, only week WEP encrypted networks.
It has some good points too, but again, the limitations above currently stop me from buying this LED TV:
- Slim, nice, beautiful menus, integrated remote control.
- Plays general simple MKV full HD movie files with high profile H.264 compression.
- As a DLNA/UPnP client, this TV works very nicely and immediately recognized and played streamed content from a Windows Media Player shared library (its a UPnP server underneath) I configured in 5 minutes on my laptop. Although I found it a a bit slow to browse the library, even connected with an ethernet cable.
I was ready to buy this piece of digital integration but left the store a bit disappointed. But I’ll keep watching their product line and hope Samsung will improve their Linux-based firmware.
© Avi for Avi Alkalay, 2010. |
Permalink |
One comment |
reactions to by Google and Technorati |
Add to
del.icio.us
Classified by author as: Multimedia
Tagged by author with:
Feed enhanced by Better Feed from Ozh
January 28, 2010 21:18
So turns out that thanks to El Al, I’ve had to change my return ticket to January 30, instead of January 23. Of course, I’m terribly sad I’ve had to extend my vacations.
I’ll just be able to post my pictures online when I return home, since I don’t want to clog up my uncle’s connection, but some pictures from the first 10 days with Taglit can be found here.
Later on I’ll write something more detailed about the whole trip.
January 28, 2010 07:49
Quem sabe a próxima geração já prefira usar iPads rodando Brushes? (Supondo sejam inquebráveis até lá.)
January 28, 2010 05:19
January 27, 2010
That’s it! The day we’ve all been waiting for finally arrived. The hype was really, well, hypey. I actually read somewhere a guy saying that this would be a day he’d be able to talk about with his grand-kids one day. What. The. Fuck. People! Seriously?
Now to be honest I was a bit hopeful, too. I mean, since that wonderful iPhone announcement a few years ago, we’re always kind of waiting for when Apple is going to do its thing again. You know, change the friggin’ world!
As it turned out, it was a historic announcement, just not the way people expected it to be. It was history because Apple finally did it: it disappointed.
I’m not saying the iPad – dude! iPad? Really, Steve? — anyway, I’m not saying the iPad is bad in and on itself. It’s a nice device. It’s just not what I was expecting. I was expecting to see something revolutionary and that the iPad is not. It is essentially a big iPod touch. It. Doesn’t. Even. Cure. Cancer!
I confess I was a lot more interested in an update to the iPhone OS and a new iLife that whatever tablet Apple was going to announce. I intentionally decided not to buy into the whole hype of the "iSlate." I thought it was going to be cool but unaffordable, which it turned out to be. I mean, $500 for an iPod touch? And the 3G model that’s essencially an iPhone without the phone? It’s $829 for an i!
With nothing new for the iPhone and no new iLife, the whole announcement was a non-event, which means it was a disappointment. A utter meh.
#apple #fail
Move along, fanbois, nothing to see here.
January 27, 2010 23:50
Things are starting to get busier on my new project. It is, alas, still listed as Restricted Secret, which means I cannot really talk about its details. Nor could I even reveal its codename, even though codenames reveal nothing about what a project really is. For the sake of making referring to it slightly less annoying for me, I’ll refer to it using a made-up codename in the best traditions of the company’s history of naming things for lakes, peaks, creeks, and towns. So the secret project shall henceforth be known as Suquia Creek (being the creek across from my house.)
Since I cannot give any details on Suquia Creek, what could I possibly talk about? A lot, it turns out. I can talk about things I’m learning while working on the project.
Ah boy, am I learning!
It turns out Suquia Creek is going to be the longest, most complex project I’ve worked on. Until SC came along, the longest project of my life had been Lava Peak, which ran for about two years. But that included post-launch activities. We managed to go from ideation to shipping in just under a quarter (I even won an award because of that back in 2007.) Suquia Creek, on the other hand, is scheduled to ship in 2014! That’s nearly half a decade of work.
As well, for earlier projects I only had to worry about software. Suquia Creek, however, is also hardware. And on the software side, off the top of my head, it involves –
- Processor µcode
- Chipset code
- BIOS extensions
- Firmware code
- Drivers
- An SDK
- User-level apps
- And a few other things I can’t say without revealing more than I should.
Bottom line: it’s huge as far as I’m concerned! And we should support multiple versions of Windows and one distro release of Linux. It also involves several cross-functional, geographically-dispersed teams based mainly in Argentina and the US, but also with some smaller efforts coming out of China, India, and Israel. That amounts to six different timezones, for a current maximum time difference of 17 hours.
And then we come back to schedule. I have no idea what I’ll be doing at home during this weekend, but I have to have a rough idea of what we’ll be delivering on, say, week 41 of 2013! That assumes the world will not end in 2012, of course.
This week we had our first engineering meeting to plan on a tentative schedule. Late next month I’ll be flying around between Silicon Valley and Silicon Forest to work out the (semi-)hard schedule. By then, we’ll actually have one internal alpha release in place already. After that, we’ll have another alpha before our first “release”, an internal proof of concept, which will then be used by customer as part of a (quasi-confidential) pilot. The customer? One of the world’s largest… well, can’t say what is their industry yet. It’s huge though.
After that pilot I expect to be able to open up a bit on what the project actually does. In the meantime, I’m going to be sharing my learning experience.
It’s going to be an exciting half-decade for me 
January 27, 2010 17:49
Hoje é meu terceiro aniversário de casamento. Na época eu disse que tinha motivos pra me casar, e ontem à noite eu finalmente peguei meu motivo de me casar: eu e Fabio conseguimos nossos vistos de imigrantes canadenses. Eu disse que 2010 ia ser um ano incrível.
Gostaria, no entanto, de dizer apenas que embora casada há “só” três anos, amanhã completamos DEZ ANOS juntos. Isso é UM TERÇO da minha vida. E foram dez anos muito felizes. Quero mais dez anos felizes assim.
Feliz Aniversário, Fabio. Everything Will Change.
:*******
Update: Para quem perguntou como eu consegui o visto, eu fiz o processo federal de skilled worker. As regras mudaram da época do meu processo pra cá, então, melhor consultar o Site da Imigração para o Canadá.
January 27, 2010 12:10
January 26, 2010
As I said in the last post, I was thinking about direct (time-domain) analysis of QAM signal at RX part, in order to allow baud rates comparable to carrier frequency. I did not plan to put that into use... but I could not resist doing that.
The thing was working sooner than expected, and performance went beyond my expectations. It can handle 2400 baud with a 1800Hz carrier, like faster modems used to do. It can currently handle 7 bits per symbol, which translates to 16800bps. Going beyond this will demand improvements in symbol detection, which is currently the same as V2 (which is suboptimal).
V3 modem is basically the same as V2, except by decodification of QAM into complex samples, which V2 did in a "classical" fashion (demodulation + FIR filtering), and V3 does by direct analysis of QAM signal. So, I will list only the relevant V3 RX part:
# Proof that we don't need exact carrier frequency
CARRIER += random.random() * 20 - 10
# Proof that we don't need exact carrier phase
ra = random.random() * 2 * math.pi
symbols = []
f = CARRIER * 2 * math.pi / SAMPLE_RATE
carrier_90 = SAMPLE_RATE / CARRIER / 4.0
carrier_90 = int(carrier_90)
phase_error = 2 * math.pi * CARRIER / SAMPLE_RATE / 2.0
recovered = []
for t in range(0, len(qam) - carrier_90 - 2):
# Take derivatives of now and 90 degrees in future
d = (qam[t+1] - qam[t]) / f
d90 = (qam[t + carrier_90 + 1] - qam[t + carrier_90]) / f
st = math.sin(f * t + ra)
ct = math.cos(f * t + ra)
a = d90 * ct + d * st
b = d90 * st - d * ct
try:
tphase = -b / a
phase = math.atan(tphase)
except ZeroDivisionError:
phase = math.pi / 2
if (-b * a) < 0:
phase = -phase
if (abs(d) > abs(d90)):
amplitude = -d / (st * math.cos(phase) + ct * math.sin(phase))
else:
amplitude = -d90 / (-st * math.sin(phase) + ct * math.cos(phase))
if amplitude < 0:
amplitude = -amplitude
phase += math.pi
phase += math.pi * 2
phase %= math.pi * 2
cpx = crect(amplitude, phase)
# print t, int(phase * 180 / math.pi) % 360, cpx
recovered.append(cpx)
This technique is based on the fact that QAM signal is generated by a very simple formula:
s(t) = m . cos(2πft + p)
where "m" and "p" are the polar coordinates of constellation symbol being currently transmitted. The derivative of that is also a simple formula:
s'(t) = 2πf.m.sin(2πft + p)
The QAM decoding problem is: given s(t), find the unknown variables "m" and "p". We need two samples of s'(t)
which are near enough to belong to the same symbol, to make an equation system and solve for "m" and "p".
That's exactly what the piece of code listed above does. The samples are taken 90 degrees apart because the equations would simplify very nicely.
I chose the s'(t) equation instead of s(t) because the QAM signal may not be centered at zero; that is, it may have some offset, or DC component. But the difference between each sample and the next is unaffected by the offset. "Difference" in a discrete signal is equivalent to "derivative" in a continous signal.
It is important to measure samples EXACTLY 90 degrees apart, which means that the "distance" between samples must be an integer number. This implies that WAV must be a multiple of 1800 / 4 = 450 Hz. Using a WAV of 43200 Hz works beautifully, while a WAV of 45000 Hz, which is better in theory and matches carrier, will not bear more than 4 bits/symbol. 44100 Hz won't work perfectly even for 3 bits/symbol.
Maybe I have "cheated" by choosing a "perfect" WAV sampling rate, but I suspect that a real-world modem does just that. Anyway, this technique of mine is not intended to be the best in town, it is just an exercise to try to go beyond "classical" QAM. Certainly there are ways to compensate for non-perfect alignment of carrier and sampling rate,
January 26, 2010 05:10
January 25, 2010
Segundo o livro "A Cabeça do Brasileiro", somos um povo familista. Familismo é dar mais valor às relações familiares do que às relações de amizade. Naturalmente, esta definição considera a "família estendida" de uma pessoa adulta (pai, mãe, irmão, cunhado, sobrinho) e não a "família nuclear" (cônjuge e filhos pequenos), já que esta última é (ou deveria ser) unida demais para sequer ser comparada a um laço de amizade.
O tal livro classifica o familismo como uma coisa ruim, pelo menos em nosso tempo, pois limita o "raio de ação" da pessoa, tanto geografica quanto culturalmente. Além disso, o laço familiar não é meritocrático (meritocracia é outro valor muito rarefeito na cabeça do brasileiro). Mas esta discussão está lá no livro. Os dois centavos de contribuição que pretendo dar em cima disso, é o caráter "reverso" que muitos brasileiros revelam em seu familismo.
O futebolista Edmundo, o Animal, teria dito uma vez que "todo jogador de futebol, logo que ganha o primeiro dinheirinho, compra uma Cherokee, uma loira e uma casa pra mãe". Não sei a frase é dele mesmo, mas ainda assim é costumeiramente atribuída a ele. Atenção ao grifo.
Por outro lado, falando ainda de jogadores de futebol, não passa uma semana sem a seguinte notícia: Decretada prisão do atleta Fulano por não pagamento da pensão alimentícia. O Romário é um dos habituées neste tipo de manchete.
Também vi em inúmeras ocasiões, nesses programas de auditório que distribuem prêmios, pessoas humildes dizendo o que iam fazer com a bolada: "comprar casa pra mãe" e"ajudar meus irmãos" estão sempre brigando pelo topo da lista de prioridades. Os sete filhos da sujeita terão de ser pacientes.
Será que deu pra enxergar um padrão aí?
Observando estes e muitos outros fatos, me parece que o brasileiro pratica um familismo invertido, que prioriza os personagens errados da família. Parece um traço cultural bonito, essa coisa de comprar casa pra mãe, mas na verdade é uma coisa feia.
Pelo menos na minha cabeça, a família é uma construção social cuja finalidade é cuidar da prole, com círculos concêntricos de responsabilidade: família nuclear, vertical, colateral, estendida, até chegar no grupo social e na nacionalidade. A responsabilidade primária é do pai e da mãe, mas se eles faltarem, haverá uma avó, uma tia, um padrinho.
Uma vez a criança tornando-se adulta, a utilidade e responsabilidade da família estão exauridas. Se os membros adultos de uma família mantém laços de amizade, isso é ótimo. Eu posso dizer, por experiência própria, que tenho muito mais afinidade com meu pai hoje, do que quando eu estava sob o pátrio poder. Mas isso é um bônus, uma agradável surpresa.
Tirante dívidas de gratidão, que não faz sentido pagar em dinheiro, pois muitas vezes são de tamanho infinito, não me sinto obrigado a coisa alguma em relação a meus pais. Minha obrigação é com meu filho enquanto incapaz; e com minha esposa que não trabalha fora para cuidar dele.
Se eu começasse a despender recursos comprando casa para a mãe, carro para a irmã, e extravagâncias assemelhadas de que a gente ouve falar, não estaria eu completamente errado em minhas prioridades? É pra frente que se anda.
O jogador de futebol que compra casa para a mãe durante sua (tipicamente breve) carreira, enquanto se esquiva de formar um patrimônio e/ou pagar pensão alimentícia, não está condenando seus filhos a um futuro sombrio?
Aliás, pensão alimentícia virou um dos bichos-papões da classe média. Todo mundo conhece uma história de horror a respeito do assunto. Um amigo recentemente contou-me que o juiz em certa ocasião arbitrou a pensão em 150% do seu salário. É certo que existem exageros, é certo que existem mulheres que fazem do títuo "ex-esposa" uma verdadeira profissão. Assim como muitos homens pagam 150 reais por mês de pensão para um filho e se consideram roubados e injustiçados. Em havendo exageros de parte a parte, acho que podemos supor que, na média, as pensões arbitradas são justas e condizentes com a obrigação moral de um pai manter sua prole.
Enfim, alguém vai estranhar estas minhas palavras, já que eu costumo afirmar que os pais não têm de "se matar" para dar luxo aos filhos. Nem estou advogando isso agora. Só acho que não faz sentido puxar o tapete da geração seguinte, e deixar de cuidar da própria vida, para fazer barretada à geração anterior.
Como diz o ditado popular, jabuti não sobe em árvore; se ele está lá, é porque alguém botou. É claro que esses comportamentos têm origem positiva, na educação. São as mães deste país, onde 1/3 dos nascidos não tem pai, que educam seus filhos para serem bons filhos, em vez de cumprirem seu papel e educar-lhes para serem bons maridos, bons pais e bons cidadãos.
January 25, 2010 05:13
As I promised, I rewrote parts of Python QAM modem to use complex numbers, which leaves code much shorter and clearer (provided that you understand complex numbers). Also, I put the common code in a separate file, to avoid redundancy in TX and RX parts. The FIR lowpass filter was improved too.
The files are m2co.py, m2tx.py and m2rx.py. All of them can be found at ttp://epx.com.br/wav folder.
Other ideas
I think I went far enough with this. I was planning to write a "version 3", but I am not sure I will. Versions 1 and 2 already make good classroom examples. FWIW I will tell what's in my mind for v3, maybe someone else picks it up.
First, the "classical" demodulation approach showed its limits in version 2; baud rate can not go above 1000 or 1100 Hz for a carrier of 1800 Hz. The FIR filter has a trade-off between frequency precision and time-domain precision. Increasing baud rate demands increasing frequency precision, but then symbols begin to smear together after filtering. In order to reach V.29 or V.32 baud rate (which is almost the same as carrier frequency), some other detection approach must be employed.
I played with some formulas, remembered some trigonometry identities at Wikipedia, and came up with one idea. Since the derivative of QAM signal is well-defined by a simple formula, and there are two unknown variables that shape the QAM signal -- amplitude and phase -- it is possible to "invert" the formula and find amplitude and phase given two derivatives taken from time-domain signal.
The distance between measured derivatives could be e.g. 90 degrees to make formula "inversion" even easier. BTW derivative of time-domain QAM signal is just the difference between two consecutive samples. If one wants to make it prettier, could add a moving average to this direct measure.
This approach worked on spreadsheet; it would probably work in a Python implementation. It seems possible to determine symbol in half carrier wavelength at the most, normally a quarter wavelength is enough, which means that decoding is possible when baud rate >= carrier.
Another advantage of this technique, or any other that analyzes the QAM signal directly, is more resilience to small carrier deviations. They would appear just as a very small but persistent phase delay in every sample, which can be ignored or compensated for.
I have no idea if this is the technique that advanced modems actually use.
January 25, 2010 02:49
January 22, 2010
Following the discussion of a "QAM modem" written in Python, let's take a look into the reception (RX) code.
Decoding the QAM signal (reliably, at least) is much more complicated than generating it, and RX has much more code. Because of that, I will explain each section separately.
First, some boilerplate code:
#!/usr/bin/env python
import wave, struct, math, numpy, random
SAMPLE_RATE = 44100.0 # Hz
CARRIER = 1800.0 # Hz
BAUD_RATE = 360.0 # Hz
PHASES = 8
AMPLITUDES = 4
BITS_PER_SYMBOL = 5
# FIR lowpass filter. This time we calculated the coefficients
# directly, without resorting to FFT.
lowpass = []
CUTOFF = BAUD_RATE * 4
w = int(SAMPLE_RATE / CUTOFF)
R = (SAMPLE_RATE / 2.0 / CUTOFF)
for x in range(-w, w):
if x == 0:
y = 1
else:
y = math.sin(x * math.pi / R) / (x * math.pi / R)
lowpass.append(y / R)
# Modem constellation (the same as TX side)
constellation = {}
invconstellation = {}
for p in range(0, PHASES):
invconstellation[PHASES - p - 1] = {}
for a in range(0, AMPLITUDES):
s = p * AMPLITUDES + a
constellation[s] = (PHASES - p - 1, a + 1)
invconstellation[PHASES - p - 1][a + 1] = s
qam = wave.open("qam.wav", "r")
n = qam.getnframes()
qam = struct.unpack('%dh' % n, qam.readframes(n))
qam = [ sample / 32768.0 for sample in qam ]
Part of the code is borrowed from TX: modulation parameters, constellation generator etc. Those things would fit better in a separate module, but I was lazy to do it :) Constellation loop generates a reverse-map dictionary, since we are doing the inverse of TX (given amplitude and phase, gimme the bits).
A new thing is the FIR filter generator. This time I did not use FFT to generate the coefficients; a closed-form expression was used instead. I am aware that this FIR filter is not the best possible (it ends abruptly at both sides), but served well enough for this toy.
# Demodulate in "real" and "imaginary" parts. The "real" part
# is the correlation with carrier. The "imaginary" part is the
# correlation with carrier offset 90 degrees. Having both values
# allows us to find QAM signal's phase.
real = []
imag = []
amplitude = []
# This proofs that we don't need to replicate the original carrier
# (at least, not in the same phase). Sometimes this random carrier
# phase makes the receiver to interpret training sequence as a
# character, but nothing serious.
t = int(random.random() * CARRIER)
# t = -1
# In the other hand, frequency must match carrier exactly; if we
# need to allow for frequency deviations, we must build a digital
# equivalent to PLL / VCO oscilator.
for sample in qam:
t += 1
angle = CARRIER * t * 2 * math.pi / SAMPLE_RATE
real.append(sample * math.cos(angle))
imag.append(sample * -math.sin(angle))
amplitude.append(abs(sample))
del qam
This was the "analog" part of the decoder. It is the most important, and the easiest to do, because trigonometry does so much for us.
Since we did a kind of AM modulation at TX, multiplying a digital signal by a sinusoid, we do the same at RX: multiply again by the sinusoid to get the original data. We don't know exactly the phase of QAM signal, and we don't need to. We just multiply by two local carriers, which are identical except by being 90 degress offset -- math.cos() and math.sin().
Note that I set "t" with a random initial number. This is a way to guarantee that local carrier will NOT be in phase with TX, as it happens in real world. The rest of RX will have to work harder because of this, naturally.
Incredible as it may seem, output of this demodulation is the X,Y position of symbol in modem constellation. The computed positions will be rotated, because local carrier is not in phase with original TX carrier, but since we have integrated the phase changes at TX, we just need to look for phase DIFFERENCES.
Finally, we "demodulated" amplitude in a separate channel, using abs(), just out of laziness, since we could obtain amplitude from real and imaginary parts.
It looks easy because we don't need to cope with frequency distortions. In practice, the local carrier would have to "tune" to the input signal, because AM demodulation depends on local carrier being EXACTLY the same frequency as modulated signal. And all transmission media (except our WAV file, of course) distort frequency in some degree.
# Pass all signals through lowpass filter
real = numpy.convolve(real, lowpass)
imag = numpy.convolve(imag, lowpass)
amplitude = numpy.convolve(amplitude, lowpass)
# After lowpass filter, all three lists show something like
# a square wave, pretty much like the original bitstream.
# If you want to see the result of early detection phase,
# uncomment this:
# f = wave.open("teste.wav", "w")
# f.setnchannels(1)
# f.setsampwidth(2)
# f.setframerate(44100)
# bla = [ int(sample * 32767) for sample in imag ]
# f.writeframes(struct.pack('%dh' % len(bla), *bla))
Actually, AM demodulation leaves a high-frequency residue in original data, so we pass our FIR filter in all signals (real, imaginary and amplitude). After this filtration, the signals will show themselves as quasi-square waves, like this:
We have three square waves like this: real, imag, and amplitude. Now we have the mission of extracting the symbols out of them. First, we need to detect phase of symbols:
# Detect phase based on real and imaginary values
phase = []
wrap = 2.0 * math.pi * (PHASES - 0.5) / PHASES
for t in range(0, len(real)):
# This converts (real, imag) to an angle
angle = math.atan2(imag[t], real[t])
if angle < 0:
angle += 2 * math.pi
# Move angle near to 2 pi to 0
if angle > wrap:
angle = 0.0
phase.append(angle)
Since our constellation does not use complex numbers, the X,Y coordinates found by demodulator are not directly useful to search for the symbol, so we need to compute actual angles, which we do using atan2().
# Find maximum amplitude based on training whistle (1s), and
# measure how much our local carrier is out-of-phase in
# relationship to QAM signal
max_amplitude = 0.0
carrier_real = 0.0
carrier_imag = 0.0
for t in range(int(0.1 * SAMPLE_RATE), int(0.9 * SAMPLE_RATE)):
max_amplitude += amplitude[t]
carrier_real += real[t]
carrier_imag += imag[t]
max_amplitude /= int(0.8 * SAMPLE_RATE)
carrier_real /= int(0.8 * SAMPLE_RATE)
carrier_imag /= int(0.8 * SAMPLE_RATE)
skew = math.atan2(carrier_imag, carrier_real)
print "Carrier phase difference: %d degrees" % (skew * 180 / math.pi)
del imag
del real
Here we analyze the "training" whistle sent by TX, in order to determine the amplitude range, as well as the phase difference between our carrier and TX carrier. We do this in a very lazy way here, and it would not be ok for a real-world modem.
Any real-world medium, be it wireless or wire or optics, does distort amplitude, phase and frequency, so it wouldn't work to estimate both at the beginning of session and leave it that way. Real-world modems use a PLL to generate local carrier, which adjusts itself all the time accordingly to input conditions.
Too good our WAV file is a perfect medium :)
# Normalize/quantify amplitude and phase to constellation steps
qsymbols = []
for t in range(0, len(phase)):
p = phase[t]
a = amplitude[t]
a /= max_amplitude
a = int(a * AMPLITUDES + 0.49)
# Compensate for local carrier phase difference
# This ensures the best phase quantization (avoiding that
# quantization edge is too near a constellation point)
p += 2 * math.pi - skew
p /= 2 * math.pi
p = int(p * PHASES + 0.49) % PHASES
qsymbols.append((p, a))
del phase
del amplitude
Here we take the "analog" phases and amplitudes, and quantize them into constellation. At this point we "unskew" the angles, so the quantified values map directly to our constellation.
The problem now is: we have thousands of samples which contain just a handful of symbols. We need to detect how many symbols are there -- and where.
# Try to detect edges between symbols
edge = []
settle_time = int(SAMPLE_RATE / BAUD_RATE / 8)
settle_timer = -1
in_symbol = 0
first_symbol = 0
last_a = last_p = 0
for t in range(0, len(qsymbols)):
s = 0
p, a = qsymbols[t]
if in_symbol > 0:
# we presume we are flying over a valid symbol
in_symbol -= 1
elif a != last_a or p != last_p:
# change detected, look for stabilization in future
settle_timer = settle_time
elif settle_timer > 0:
# one sample closer a good symbol
settle_timer -= 1
elif settle_timer == 0:
# signal settled for (settle_time) samples;
# take the current signal as a valid symbol
settle_timer = -1
in_symbol = int(0.85 * SAMPLE_RATE / BAUD_RATE)
if t > 0.5 * SAMPLE_RATE:
# make sure we are not at the beginning
s = 1
if not first_symbol:
first_symbol = t
edge.append(s)
last_p = p
last_a = a
At this section, we count on the fact that most symbols show a distinctive "edge" between them. We annotate the points where this edge is present, for further reference.
Note that this step did not find all symbols, because the message may have repeated symbols with phase 0, which don't s how any discontinuity. This is a direct result of poor constellation choice; it would have been better to avoid phase-0 points, so we would find all symbols by looking for edges.
# Find symbols based on edges and known baud rate
symbols = []
expected_next_symbol = SAMPLE_RATE / BAUD_RATE
lost = 0
for t in range(first_symbol - int(SAMPLE_RATE * 0.1), len(qsymbols)):
p, a = qsymbols[t]
s = edge[t]
if s:
# amplitude/phase edge, strong hint for symbol
lost = 0
symbols.append([p, a])
else:
lost += 1
if lost > expected_next_symbol * 1.5:
# no edge yet, take this as a non-transition signal
lost -= expected_next_symbol
symbols.append([p, a])
del qsymbols
del edge
Here we finally detect the handful of symbols we are interested in, and we can throw out the enormous time-domain lists once and for all.
The strategy of symbol detector is simple. Every detected edge is a new symbol for sure. But, if no edge was found after some time (baud rate + 50% tolerance), we blindly assume that we have a repeated symbol there.
Even if TX and RX baud rates were a bit different, this strategy would be robust, because most symbols do have an edge between them, so the algorithm will not have to work blind for more than 2 or 3 symbols at a time.
print "%d symbols found" % len(symbols)
# Differentiate phase (because we had integrated it at transmitter)
last_phase = 0
for t in range(0, len(symbols)):
p = symbols[t][0]
symbols[t][0] = (p - last_phase + PHASES) % PHASES
last_phase = p
In TX we had integrated (summed up) the phase offsets, so now we have to do the opposite operation.
# Translate constellation points into bit sequences
bitgroups = []
for t in range(0, len(symbols)):
p, a = symbols[t]
try:
bitgroups.append(invconstellation[p][a])
except KeyError:
print "Bad symbol:", p, a, ", ignored"
# Translate bit groups into individual bits
bits = []
for bitgroup in bitgroups:
for bit in range(0, BITS_PER_SYMBOL):
bits.append((bitgroup >> (BITS_PER_SYMBOL - bit - 1)) % 2)
Symbol characteristics are traded for bit sequences, and finally we can grab the actual bits we were looking for. The modem demodulation part is done, but now we have to recover bytes from bits, like a RS-232 interface would do.
# Find bytes based on start and stop bits. This will make you feel
# like a poor RS-232 port :)
t = 0
state = 0
byte = []
bytes = []
while t < len(bits):
bit = bits[t]
if state == 0:
# did not find start bit
if not bit:
# just found start bit
state = 1
elif state == 1:
# found start bit, accumulating a byte
if len(byte) < 8:
byte.append(bit)
else:
# byte complete, test stop bit
if bit:
# stop bit is there
bytes.append(byte)
else:
# oops, stop bit not there, we were cheated!
# backtrack to the point fake start bit was 'found'
t -= 8
byte = []
state = 0
t += 1
# Make a string from the bytes
msg = ""
print "Bytes:",
for byte in bytes:
value = 0
for bit in range(0, 8):
value += byte[bit] << (8 - bit - 1)
msg += chr(value)
print value,
print
print msg
The serial-to-bytes loop looks for start bits (zeros) and stop bits (ones). Once it finds a bit sequence that satisfies these conditions, it cuts a byte. If either the start or stop bit is missing, it keeps reading bits until it gets synchronized again.
Amazingly, this thing works. The improvements I can imagine for this Python modem are:
1) Take a closer look at the FIR filter, choosing the best cutoff frequency and avoid abrupt ends;
2) Choose a better constellation, without 0-phase points, and use complex numbers;
3) Generate RX local carrier using a PLL (phase-locked loop), so it can "follow" frequency and phase distortions.
4) Measure amplitude range continually using AGC (automatic gain control)
5) Add a bit randomizer
January 22, 2010 20:33
If you are an old-timer computer guy like me... you are probably nostalgic of this noise:
http://epx.com.br/wav/qam_sample.wav
After playing with analog modulations, this time I wanted to play with some digital modulation. I thought about PSK and FSK, but they are quite simple, and are just particular cases of QAM -Quadrature Amplitude Modulation. The audio abovementioned was generated by my "modem", written in Python.
In a nutshell, writing the transmission (TX) part of a modem is easy. The big challenge is to write the reception (RX) part. As you will see, the RX code is far longer and more complicated.
I did not try to write the perfect modem, so this thing will probably not work over a ham radio link or any other non-ideal transmission medium. I will point out the shortcomings that I know of, and there are for sure a whole bunch of other problems that I don't even know of. It's just to have fun, after all.
QAM modulation takes a carrier of fixed frequency and changes phase as well as amplitude, accordingly to the bits that are to be sent. The output wave is something like this:
Note that QAM, like AM, never changes carrier frequency. It would not be possible to change frequency along with phase and amplitude, because they would smear each other. (The digital cousin of FM is FSK).
Each change in QAM signal may carry several bits. For example, if we establish that we have 4 possible phases (0, 90, 180 and 270 degrees), and 2 possible amplitudes (50% and 100%), we have 8 possible combinations, which allow us to send 3 bits per sample (log2(8)).
So we have this concept of "baud rate", which is the number of symbol changes per second. The data bit rate is the baud rate multiplied by the number of bits that each symbol can carry.
Of course, we need to establish a relationship between phases, amplitudes and transmitted bits in an orderly manner. The nost common way to do it is using a "constellation" graph like this:
In the image above, we have 3 constellations. At left, a very simple one, with only four points. If you draw a line from origin (coordinates 0,0) to a given point, the line length is the signal amplitude, and line angle is the phase change. We just need to attribute a different bit pattern for each point.
Since all four points of the leftmost constellation are equidistant from origin, we can say that this constellation is not really QAM; it is PSK (phase shift keying) because it only uses phase changes to send information The other constellations are more cluttered, and have points of several amplitudes.
WIthout further delay, here is the code of modem TX part:
#!/usr/bin/env python
import wave, struct, math
SAMPLE_RATE = 44100.0 # Hz
CARRIER = 1800.0 # Hz
BAUD_RATE = 360.0 # Hz
PHASES = 8
AMPLITUDES = 4
BITS_PER_SYMBOL = 5
# generate constellation of phases and amplitudes
# for each possible symbol, and attribute it to a
# certain bit pattern. I avoided deliberately the
# complex numbers, even though they make things
# easier when dealing with constellations.
#
# Make sure that all ones = maximum aplitude and 0 phase
constellation = {}
for p in range(0, PHASES):
for a in range(0, AMPLITUDES):
s = p * AMPLITUDES + a
constellation[s] = (PHASES - p - 1, a + 1)
# This is the message we want to send
msg = "Abracadabra!"
# msg = open("modem_tx.py").read()
# Convert message to a list of bits, adding start/stop bits
msgbits = []
for byte in msg:
byte = ord(byte)
msgbits.append(0) # start bit
for bit in range(7, -1, -1): # MSB
msgbits.append((byte >> bit) % 2)
msgbits.append(1) # stop bit
# Round bit stream to BITS_PER_SYMBOL
while len(msgbits) % BITS_PER_SYMBOL:
msgbits.append(0)
# Group bit stream into symbols
msgsymbols = []
for n in range(0, len(msgbits), BITS_PER_SYMBOL):
symbol = 0
for bit in range(0, BITS_PER_SYMBOL):
symbol += msgbits[n + bit] << (BITS_PER_SYMBOL - bit - 1)
msgsymbols.append(symbol)
# Add 1 second worth of all-1 bits in header, plus a trailer.
# The 11111 bit pattern generates a pure carrier, that receiver
# can recognize and train.
allones = (1 << BITS_PER_SYMBOL) - 1
train_sequence = [ allones for x in range(0, int(BAUD_RATE)) ]
finish_sequence = [ allones, allones ]
msgsymbols = train_sequence + msgsymbols + finish_sequence
qam = wave.open("qam.wav", "w")
qam.setnchannels(1)
qam.setsampwidth(2)
qam.setframerate(44100)
t = -1
phase_int = 0
samples = []
for symbol in msgsymbols:
amplitude = 0.9 * constellation[symbol][1] / AMPLITUDES
# Phase change is integrated so we don't need absolute phase
# at receiver side.
phase_int += constellation[symbol][0]
phase_int %= PHASES
phase = phase_int / (0.0 + PHASES)
# It would be easier to manipulate phase using complex numbers,
# but I thought this way would be clearer.
# Play the carrier at chosen phase/amplitude for 1/BAUD_RATE seconds
for baud_length in range(0, int(SAMPLE_RATE / BAUD_RATE)):
t += 1
sample = math.cos(CARRIER * t * math.pi * 2 / SAMPLE_RATE \
+ 2 * math.pi * phase)
sample *= amplitude
samples.append(int(sample * 32767))
qam.writeframes(struct.pack('%dh' % len(samples), *samples))
print "%d symbols sent" % len(msgsymbols)
Actually, the most complicated part of TX is the bit stream expansion. The original message is supplied as a string of bytes, so we need to "explode" it to individual bits, and add start and stop bits. The output signal itself is easy to generate, courtesy of math.cos() function :)
Do you remember that modems used to "whistle" for a couple seconds before connection? It was necessary to do the same here, so the receiver can get "trained" to the signal. Despite the fact the thing reads data from a high-quality sound file, the training phase is still needed.
I employed a 1800 Hz carrier just out of nostalgy, because this resembles the pitch of once-ubiguitous telephone modems.
The other parameters were found empirically; they are just below the level in which RX code fails to recover the message.
The constellation I generate would plot like a bicycle wheel, with "spokes" coming from origin to the outer rim. This is NOT an optimal constellation at all, because it is more cluttered near the origin, which means that those low-amplitude symbols are more easily misinterpreted in presence of noise. The constellations shown in second picture have better scattering.
There is another reason why my constellation is sub-optimal. I opted not to use complex numbers, to keep the code more readable for non-mathematically-inclined programmers. Unfortunately, making an optimal constellation without complex numbers means unreadable code too, so I chose to keep it simple and sub-optimal. (I plan to write a version of this QAM code using complex numbers in a future post.)
Yet another potential problem of my constellation, is that there are 0-degree points, meaning a symbol that does not change phase. This is difficult on RX because if such a symbol is sent repeatedly, the QAM signal will not have any "bump" between symbols, and RX will have to guess the symbol size. Another thing to fix in a new modem version.
Talking about phase, the phase change is integrated at TX side. For example, if you have several 90-degree symbols in sequence, the angles are accumulated, so the output signal will have 90, 180, 270, 0, 90, 180... phased symbols. At RX side, the phase CHANGE is taken into account, so the symbols are correctly interpreted as 90, 90, 90, 90... If phase were not integrated at TX, the RX side would have to know the ABSOLUTE phase all the time, and would lose forever the sync as soon as the first noise struck the signal.
Note that amplitude, by its nature, does not need to be integrated to be correctly interpreted by RX. A simple moving average at RX side is enough to know the amplitude range.
The bit stream can also be conditioned by a "randomizer". It avoids long sequences of 0s and 1s, thus avoiding long sequences of a single symbol, and making RX life easier. I did not implement a randomizer in this version, but perhaps I'll do in the next one.
January 22, 2010 16:15
Even when working at Layer 2, ExtremeWare takes
advantage of information available at other layers to
establish and enforce QoS and access control policies.
Layer-independence means that when an Extreme Networks
product is routing or switching packets, QoS and access
control decisions can still be made using criteria pulled from
Layers 1–4 or from other sources.
(Source: «Extreme Networks Technical Brief: ExtremeWare Operating System»)
January 22, 2010 13:57
Seguindo o hype e o movimento de mudança de vários amigos e da internerds, resolvi experimentar o Chrome como meu browser principal. Migrei as abas abertas, fechei o firefox e deixei apenas o novo navegador do google no meu desktop (na verdade utilizei o Chromium, versão opensource do Google Chrome, como disponibilizado para o Fedora Linux 12).
No início fiquei muito impressionado com a velocidade do Chrome e me senti de certo modo aliviado em deixar o Firefox. Repeti algumas vezes pra quem estava próximo: “o Firefox está com os dias contados, não tem a mínima chance”.
Mas depois de 15 dias, estou voltando pro Firefox 3.5. Os ganhos em velocidade do Chrome não foram suficientes pra compensar as inconveniências e a (falta das) boas extensões do concorrente. O navegador do Google ainda tem que amadurecer um pouco até se tornar o browser definitivo.
O que eu gostei no Chrome:
- Rápido, muito rápido. Da inicialização aos sites carregados de java-script, o Chrome dá de lavada no Firefox;
- A ideia de um processo por aba/extensão e o isolamento entre eles me agrada;
Existem outras características e mudanças que são boas, mas que não fizeram diferença considerável no uso do browser.
Os principais inconvenientes, em ordem de prioridade:
- Não suporta master-password;
Pra colar uma nova URL, é preciso clicar na barra de endereços (em outros browsers no Linux é só clicar com o botão do meio – colar – em uma área livre do browser); update: é só clicar com o botão do meio no botão de nova aba (thanks to Andreas)- O Chrome usa $LC_CTYPE pra definir o idioma (o correto é usar $LC_MESSAGES ou $LANG);
- Force-reload não funciona como esperado;
- Compatibilidade com sites: foram poucos problemas, mas notavelmente mais do que com o Firefox. Alguns exemplos de sites que apresentaram um ou outro problema: gvt.com.br, mercadolivre.com.br, voeazul.com.br;
- Entre a omnibox do Chrome e a awesome bar do Firefox, eu prefiro esta última combinada com uma caixa de busca;
- Extensões, sempre elas: deixei esse item por último, mas talvez seja o que mais impactou minha produtividade. Pra mim as que fizeram mais falta foram: tabmixplus, echofon/twitterfox, delicious bookmarks, multiproxy switch, dns cache e web developper. Eu tentei substituí-las pelas alternativas do Chrome, mas estas ainda são muito fracas.
O que eu aprendi com essa experiência é que, contrário do que eu achava originalmente, trocar um browser não é tão simples como parece e o Chrome ainda tem muito chão pela frente até se tornar um browser maduro. Além disso também não vejo mais o Chrome como uma ameaça tão séria ao Firefox: levará pelo menos mais um ano até ele se tornar tão usável quanto e, até lá, quem sabe o Firefox 4.0 já esteja nas ruas incorporando suas principais características positivas.
De qualquer modo, é bom ver o aparecimento de uma nova “guerra dos browsers”. Tomara que dessa vez não haja um vencedor.
Posts possivelmente relacionados:
- Primeiros dias de trabalho em Angola
- De volta ao Brasil
- De volta ao trabalho
Link Permanente |
Nenhum comentário
Post tags: computação, google, linux
January 22, 2010 12:12
January 21, 2010
FM (Frequency Modulation) is the mainstream technology employed in hi-fi analog radio. In this post, I will try to show how a signal "sounds" once it is modulated this way, and of course show a simple implementation in Python.
We have this original speech of mine as input signal -- the same I used for AM modulation, and probably a lot of people can't stand hearing it once more :)
paralelepipedo_lopass.wav
Now I modulate to FM, using a carrier of 5512.5 Hz (which happens to be 44100/8) and modulation band of +/-1000Hz. Please don't hear it in high volume, otherwise it will blow your head!
fm.wav
Impossible as it may seem, the voice can be recovered from the modulated version:
demod_fm.wav
If your browser can't play audio files using embed tag, you can find the audios at http://epx.com.br/wav/.
Now, let's see how the sausage is stuffed. First, a bit of background explanations.
FM changes the frequency of a central carrier, based on input signal strength. Stronger signals will cause farther deviations in modulation, regardless of input signal frequency. The carrier output is always the same power. In order to limit the minimum and maximum carrier frequencies, the strength of input signal must be bounded.
This is completely different from AM, where the carrier power oscilates to mimick the input signal, but the AM carrier frequency is fixed.
FM is much more robust against noise, because carrier output is always at maximum power, while AM output may be overwhelmed by noise when it is in a "valley". In the other hand, FM needs more bandwidth than AM to transmit the same signal.
Here is the FM modulator written in Python, very simple indeed:
#!/usr/bin/env python
import wave, struct, math
baseband = wave.open("paralelepipedo_lopass.wav", "r")
FM_BAND = 1000.0
FM_CARRIER = 44100 / 8.0
fm = wave.open("fm.wav", "w")
fm.setnchannels(1)
fm.setsampwidth(2)
fm.setframerate(44100)
integ_base = 0
for n in range(0, baseband.getnframes()):
base = struct.unpack('h', baseband.readframes(1))[0] / 32768.0
# Base signal is integrated (not mandatory, but improves
# volume at demodulation side)
integ_base += base
# The FM trick: time (n) is multiplied only by carrier freq;
# the frequency deviation is added afterwards.
signal_fm = math.cos(2 * math.pi * FM_CARRIER * (n / 44100.0) +
2 * math.pi * FM_BAND * integ_base / 44100.0)
fm.writeframes(struct.pack('h', signal_fm * 32767))
Since the input signal influences the frequency directly, it goes "inside" the cosine operation. The major trick is: time multiplies the base carrier part ONLY. I had a hard time making the modulator to work, because I was multiplying everything by "n" and the output was white noise. In the end, Wikipedia saved me :) Perhaps it wouldn't make a difference if carrier frequency was well above the input signal, but in this example they are fairly close (1Khz to 5KHz).
FM modulation looks simpler than AM (less lines of code, at least); and it is! A FM transmitter can be build with a few cheap components. Of course the high frequency of commercial FM helps in this too: no big coils etc.
In the other hand, FM demodulation is tricky; even in DIY FM receiveres the demodulation part is carried out by a custom integrated circuit, some TDAxxx. There are several methods of FM demodulation. I have "invented" a new method for my Python FM demodulator. It is grossly suboptimal, but it works:
1) pass FM signal through a ramp/bandpass filter, from FM_CARRIER-FM_BAND to FM_CARRIER+FM_BAND. Signals near -FM_BAND are made weaker and signals near +FM_BAND are made stronger, in a proportional way.
This converts the FM signal into a form of AM modulation, because the processed signal strength shows relationship with original signal's strength.
2) detector and low-pass filter, exactly as in an AM receiver.
Here is the code. It is quite long because I employed two FFT filters, one at FM-AM conversion, and another as a low-pass filter. The filters' "technology" came directly from the FFT filter post.
#!/usr/bin/env python
SAMPLE_RATE = 44100 # Hz
FM_BAND = 1000.0
FM_CARRIER = 44100 / 8.0
LOWPASS = FM_CARRIER - FM_BAND
HIGHPASS = FM_CARRIER + FM_BAND
HIGHPASS2 = FM_BAND # Hz
import wave, struct, math
from numpy import fft
FFT_LENGTH = 2048
OVERLAP = 512
FFT_SAMPLE = FFT_LENGTH - OVERLAP
NYQUIST_RATE = SAMPLE_RATE / 2.0
LOWPASS /= (NYQUIST_RATE / (FFT_LENGTH / 2.0))
HIGHPASS /= (NYQUIST_RATE / (FFT_LENGTH / 2.0))
HIGHPASS2 /= (NYQUIST_RATE / (FFT_LENGTH / 2.0))
zeros = [ 0 for x in range(0, OVERLAP) ]
# Make ramp filter for FM-AM conversion
mask = []
for f in range(0, FFT_LENGTH / 2 + 1):
if f < LOWPASS or f > HIGHPASS:
ramp = 0.0
else:
ramp = (f - LOWPASS) / (HIGHPASS - LOWPASS)
mask.append(ramp)
# make lowpass filter
mask2 = []
for f in range(0, FFT_LENGTH / 2 + 1):
if f > HIGHPASS2 or f == 0:
ramp = 0.0
else:
ramp = 1.0
mask2.append(ramp)
fm = wave.open("fm.wav", "r")
demodulated = wave.open("demod_fm.wav", "w")
demodulated.setnchannels(1)
demodulated.setsampwidth(2)
demodulated.setframerate(SAMPLE_RATE)
n = fm.getnframes()
fm = struct.unpack('%dh' % n, fm.readframes(n))
# scale from 16-bit signed WAV to float
fm = [s / 32768.0 for s in fm]
saved_td = zeros
intermediate = []
# Convert FM to AM
for pos in range(0, len(fm), FFT_SAMPLE):
time_sample = fm[pos : pos + FFT_LENGTH]
frequency_domain = fft.fft(time_sample, FFT_LENGTH)
l = len(frequency_domain)
for f in range(0, l/2+1):
frequency_domain[f] *= mask[f]
for f in range(l-1, l/2, -1):
cf = l - f
frequency_domain[f] *= mask[cf]
time_domain = fft.ifft(frequency_domain)
for i in range(0, OVERLAP):
time_domain[i] *= (i + 0.0) / OVERLAP
time_domain[i] += saved_td[i] * (1.0 - (i + 0.00) / OVERLAP)
saved_td = time_domain[FFT_SAMPLE:]
time_domain = time_domain[:FFT_SAMPLE]
intermediate += time_domain.real.tolist()
# Detector "diode"
intermediate = [ abs(sample) for sample in intermediate ]
saved_td = zeros
output = []
del fm
# Filter the result, removing high frequencies
for pos in range(0, len(intermediate), FFT_SAMPLE):
time_sample = intermediate[pos : pos + FFT_LENGTH]
frequency_domain = fft.fft(time_sample, FFT_LENGTH)
l = len(frequency_domain)
for f in range(0, l/2+1):
frequency_domain[f] *= mask2[f]
for f in range(l-1, l/2, -1):
cf = l - f
frequency_domain[f] *= mask2[cf]
time_domain = fft.ifft(frequency_domain)
for i in range(0, OVERLAP):
time_domain[i] *= (i + 0.0) / OVERLAP
time_domain[i] += saved_td[i] * (1.0 - (i + 0.00) / OVERLAP)
saved_td = time_domain[FFT_SAMPLE:]
time_domain = time_domain[:FFT_SAMPLE]
output += time_domain.real.tolist()
# Scale signal and write to WAV file
output = [ int(sample * 32767) for sample in output ]
demodulated.writeframes(struct.pack('%dh' % len(output), *output))
January 21, 2010 05:14
January 20, 2010
No Mercado Livre há um "mar" de modems ADSL Thomson SpeedTouch ST510V6 para vender, e sempre por um preço muito baixo. Comprei um esses dias por R$ 15.
Não sei exatamente qual a razão do baixo preço (seriam modems em comodato?), mas uma desvantagem óbvia deste modem para um leigo é que não se pode configurá-lo via Web. Ou ele está configurado para sua operadora, ou não está. Mesmo em modo bridge, é preciso pelo menos configurar os parâmetros VPI e VCI da camada ATM, que variam conforme a operadora. Assim, um modem SpeedTouch pré-configurado está, na prática, "bloqueado".
Ele até tem modo telnet, que em tese permite mexer nestas configurações, mas a interface CLI é horrível. Embora eu tenha conseguido configurá-lo para usar apenas ADSL2, não consegui alterar VPI e VCI por este caminho.
Mas como tudo tem jeito exceto a morte e os impostos, é possível reconfigurar o modem usando um software externo, denominado "STInstall". Encontrei o tal programa seguindo a pista do site http://blog.mhavila.com.br/2006/09/17/roteador-adsl-speedtouch-510v6/.
Não foi muito fácil achá-lo pois a maioria dos links que oferecem o tal aplicativo estão quebrados. O seguinte link parece estar funcionando enquanto escrevo isto: http://speedtouch.com.br/st510v6/SpeedTouch510v6.zip. Parece que o programa também está disponível via Torrent.
O site brasileiro http://www.speedtouch.com.br não fornece o STInstall, mas apenas o STUpgrade, que serve para atualizar firwmare (mantendo a configuração "bloqueada" original).
Aparentemente, o STInstall era oferecido por uma telecom belga (que preferiu dar a seus usuários a possibilidade de reconfigurá-lo). Não é a primeira vez que um programa de telecom estrangeira salva o nosso dia; o modem 3G ZTE MF622 da Claro funciona com o "discador" Mac OS X da australiana Telstra (a Claro fornece o "discador" em CD mas não o disponibiliza para download em lugar nenhum, o que complica a vida se você não está de posse do CD).
Importante!
O programa é para Windows, mas funcionou bem no VMWare, colocando a rede em 'bridge'. Não vou colocar um link para download a partir do meu site pois realmente não sei qual é a situação de licença deste programa.
Além disso, ele sempre pode estar contaminado por algum keylogger ou coisa do gênero. Como eu só uso o Windows em VMWare, e só para esse tipo de bobagem, aceitei correr um risco calculado para fazer meu modem funcionar.
Se você usa Windows como sistema operacional principal, talvez fosse bom criar uma VM ou usar um outro computador só para reconfigurar o modem.
E o modem presta?
Diz a lenda que é um dos poucos modems que funciona bem em modo router, sem travar etc. Como eu tenho roteador Linux, uso sempre em modo bridge. Nos poucos testes que fiz, o meu modem "caro", o SpeedStream, parece ter desempenhado melhor. Adquiri o SpeedTouch para ter um modem-reserva que fosse ADSL2.
Uma curiosidade: este modem usa uma fonte de 22V, com um plug ligeiramente diferente do normal. Se for comprar um no ML, certifique-se que vem com a fonte, pois certamente suas fontes sobressalentes são de 5V ou 12V, mas não de 22V.
January 20, 2010 14:17
January 19, 2010
As peças para o gabinete chegaram na segunda pela manhã, e corri para casa no final do expediente para poder brincar pelo menos um pouco com elas. Montei tudo, espalhado mesmo, sobre a mesa da cozinha aproveitando um teclado e mouse velhos que achei em um canto e o monitor LCD do micro do Gabriel. Para ver se funcionava, tasquei um pendrive com o Ubuntu 9.10. E não é que funcionou de primeira?
Gostei do desempenho do Atom 330 no geral: a máquina é silenciosa e “esperta”, responde rapidinho sem te deixar esperando. Infelizmente, os testes com o SDLMame desapontaram: o desempenho em um Atom dual-core não foi muito superior ao em um Atom single-core (no meu Dell Mini 9): Neo*Geo roda a 100% com frameskip zero, mas Out Run chega a só 60% da velocidade (e som sempre ruim).
O problema aqui é o MAME, que não é otimizado para tirar proveito de múltiplos cores. Até existe uma opção para multithreading, mas ela faz pouca diferença. Na verdade, o MAME não é otimizado no geral: a abordagem dos desenvolvedores é valorizar a precisão sobre o desempenho, com o objetivo de emular o hardware original da forma mais exata possível, não importa o “custo”. Se há duas formas de fazer uma coisa, uma precisa e a outra 200% mais rápida, os desenvolvedores vão optar pela precisão.
Mas voltando ao meu arcade, a solução é diversificar: estou pensando em deixar o MAME para os jogos mais antigos (Galaga, Bosconian, Pac-Man, New Rally-X) e usar o Final Burn Alpha para os mais recentes e pesados (jogos de System 16 como Out Run, Neo-Geo, Capcom CPS-1 e CPS-2). O problema é que a versão Linux do Final Burn Alpha parece estar abandonada: não há binários pré-compilados para nenhuma distribuição e a versão mais recente do código-fonte que encontrei data de 2007 e vem acompanhada de avisos como “é velho, é instável, é beta, mesmo se compilar não vale a pena”. Existe uma versão Linux/SDL recente, que foi portada para o Dingoo, mas não sei se seria trivial levar ela de volta ao PC. Na pior das hipóteses, a solução é basear o arcade em Windows. Pelo menos emuladores não faltam.
O próximo passo é achar um gabinete para a placa, de modo que eu possa deixar todo o hardware montado sem medo de um gato resolver dormir em cima dele. Pretendo fazer um gabinete “interino” em MDF para abrigar a placa, HD e fonte. Com isso posso deixar ele no rack da sala e usar a TV de 32″ como monitor. Depois de definir o software o próximo passo é construir os controles (inicialmente como “joysticks arcade”) e por fim criar um gabinete para abrigar tudo. Isso vai acabar dando mais trabalho do que imaginei. Por enquanto, fiquem com algumas fotos da minha mesa da cozinha ontem à noite. As laranjas não fizeram parte da máquina.
January 19, 2010 17:13
Amanhã é o grande dia. Estou indo para Porto Velho (de avião) onde irei receber minha moto que está à caminho, pelo Rio Madeira. Infelizmente tamanha logística foi necessária porque a única conexão que Manaus tem com o resto do Brasil (exceto Roraima) é por barco ou avião.
Da capital rondoniense, eu e o Milton partiremos [...]
January 19, 2010 00:31
January 18, 2010
And now, for completeness, let's have an IIR (infinite Impulse Response) filter. Those digital filters calculate the output signal based not only on past input signals, but also on past OUTPUT signals. In other words, there is a feedback component on IIR output. This filter may have output forever, even if the input signal is cut, hence the "infinite" quality.
An IIR filter can be expressed by a difference equation, which defines output in terms of previous outputs, as well as inputs. A very popular example of difference equation is the Fibonacci sequence:
Fib(n) = Fib(n-1) + Fib(n-2)
In digital filtering, we are more interested in difference equations that are stable, that is, tend to zero as the input fades away. Since a functional IIR filter can be defined by a difference equation of just two terms, like the Fibonacci series above, it can do much more using much less math. In this sense, IIR is much more powerful than FIR.
The following example is a kind of resonator that enhances frequency response around 2000Hz (any other value could be chosen just by changing the source code). The "r" factor is the resonance power. If you make r=1, it becomes an oscilator tuned to 2000Hz. If you make r > 1, it becomes exponential and useless.
#!/usr/bin/env python
import wave, struct, math
SAMPLE_RATE = 44100
original = wave.open("NubiaCantaDalva.wav", "r")
filtered = wave.open("NubiaFilteredIIR.wav", "w")
filtered.setnchannels(1)
filtered.setsampwidth(2)
filtered.setframerate(SAMPLE_RATE)
# IIR filter coefficients
freq = 2000 # Hz
r = 0.98
a1 = -2.0 * r * math.cos(freq / (SAMPLE_RATE / 2.0) * math.pi)
a2 = r * r
filter = [a1, a2]
print filter
n = original.getnframes()
original = struct.unpack('%dh' % n, original.readframes(n))
original = [s / 2.0**15 for s in original]
result = [ 0 for i in range(0, len(filter)) ]
biggest = 1
for sample in original:
for cpos in range(0, len(filter)):
sample -= result[len(result) - 1 - cpos] * filter[cpos]
result.append(sample)
biggest = max(abs(sample), biggest)
result = [ sample / biggest for sample in result ]
result = [ int(sample * (2.0**15 - 1)) for sample in result ]
filtered.writeframes(struct.pack('%dh' % len(result), *result))
You can see that our IIR filter can be directly derived from frequency and resonance factor, and it has only two weights (a1 and a2), so it needs only 2 multiplications per sample, while in FIR we needed dozens, or hundreds.
Another thing you may notice is the "biggest" variable. IIR filters are quite the opposite of FIR filters; they can only increase frequency response in a certain range, while FIR can only decrease it. So, in IIR filter, we let it increase (a lot) the response around 2000Hz (above the clipping limits) and then scale all values back to avoid clipping. The net effect is of course damp all frequencies except those around 2000Hz.
The IIR filter coefficients are NOT the impulse response like it were in FIR; so you can NOT find the filter weights using an inverse FFT as we did in FIR. The impulse response of an IIR filter can be found by FFT, but this is not directly useful for actual time-domain processing.
In fact, IIR filters are VERY difficult to calculate and prone to instability problems due to feedback. Since at least the two-order IIR filter is easy to make, as we did above, there are methods to make higher-order IIR filters as a bunch of two-order IIR sub-filters, connected in series or parallel.
Another way (that I have heard of) to design IIR filters is by mimicking some well-known analog filter design. In fact, all analog filters have "infinite response" and rely on feedback (even though analog circuits have continous output), so they match IIR features well.
All software-defined filters I have seen are based on FIR. Maybe embedded things like guitar pedals are using IIR, but I can't tell. Since I don't possess the needed math to realize more complex IIR filters, and the FFT->FIR weights method worked so well, I'd go FIR if I needed to design a filter right now...
January 18, 2010 23:38
Wanderlust (wl) is the best IMAP MUA I’ve ever found. However, it takes too long to filter (“refile”) server-side messages. I wonder if it tries to download them to search?
This problem is even more annoying because wl is an Emacs application, and the Emacs OS lacks multitasking (they tell me that’s because it lacks closures, and that is because its flavour of Lisp uses dynamic scoping, and that’s hard to change). So your whole OS stays blocked while wl filter your messages.
This same issue killed any hope of a pure-Emacs web browser (like w3). People have circumvented it by calling external, non-Emacs applications, which can run asynchronously thanks to the host OS (like w3m-el).
You can do the same to organize IMAP mailboxes with this nifty little program called imapfilter. It uses Lua for configuration and knows how to filter efficiently, by asking the server to match patterns and move messages. It only deals with message IDs. The thing is so fast, it’s usually done almost instantaneously.
Here’s how to hook up imapfilter to wl:
;; on your wl configuration file, or wherever you keep this stuff
(when (and (file-readable-p "~/.imapfilter/config.lua")
(executable-find "imapfilter"))
(eval-after-load 'wl
'(add-to-list 'wl-folder-check-entity-pre-hook
(lambda ()
(message "calling imapfilter…")
(if (eq (call-process "imapfilter") 0)
(message "imapfilter ran fine.")
(message "error running imapfilter!"))))))
I use the when condition for robustness, because I copy my emacs conffiles to several machines. The eval-after-load prevents startup delay. The meat of it is the add-to-list hook, which will cause imapfilter to be called whenever you synchronize folders (key s on folder view).
January 18, 2010 17:22
As coisas não saíram exatamente como o esperado, e as peças para a máquina de arcade não chegaram na sexta-feira. Com isso, os testes que eu tinha planejado para o fim de semana com o hardware “real” foram por água abaixo. Mas nem tudo foi perdido.
Aproveitei a manhã de sábado para visitar algumas lojas na vizinhança de casa em busca de chapas de MDF para um gabinete improvisado, mas não tive sorte. E a preguiça me impediu de me aventurar mais do que alguns quarteirões além de casa, portanto a ida à Leroy Merlin mais próxima ficou para depois.
Mas instalei uma versão recente do MAME (SDLMame) em meu netbook (um Dell Mini 9 com Atom de 1.6 GHz e 1 GB de RAM) para alguns testes de desempenho. Afinal, a placa que escolhi para usar no gabinete é bem similar: mesma quantidade de RAM, mesmo chipset de vídeo (GMA 950) e mesmo clock, mas dual-core. O que rodar no netbook vai rodar melhor no gabinete.
De início, um susto: o desempenho era horrível e não consegui som no emulador, que insistia em congelar sempre que eu tentava sair dele (e dá-lhe kill -9!). Out Run (um dos jogos que pretendo emular) rodava a segundos por quadro em vez de quadros por segundo, e mesmo os jogos de Neo-Geo (como Metal Slug) estavam “injogáveis”. Tinha algo muito errado na história.
No fim, fuçando pela internet descobri que o problema é com o sistema de som usado nas versões mais recentes do Ubuntu, o PulseAudio. Para resolver o problema basta instalar um pacote extra, como mostrado neste fórum. Problema resolvido! Metal Slug agora roda perfeitamente, e Out Run chega a 43 FPS em média, sem frameskip (mas com som meio engasgado). Espero que chegue a 60 FPS na máquina dual-core.
Os componentes que encomendei chegaram nesta segunda-feira de manhã. Hora de liberar um espaço em casa para montar tudo e começar testes com o hardware “real”. Notaram que ainda não tenho planos para o gabinete, nem idéia de que monitor usar? Pois é, adotei uma abordagem “de dentro pra fora” e estou decidindo aos poucos.
January 18, 2010 15:25
January 17, 2010
Resuming from last post about DFT audio filter, I mentioned that there were two other major methods do create a digital filter: FIR and IIR. This time I will demonstrate a FIR (Finite Impulse Response) filter.
First, we need to resort to some math. We have that operation called CONVOLUTION, which can be understood as a kind of "graphical multiplication", or as a moving, weighted average. Consider the following equation:
average stock index(n) = index(n)*0.5 + index(n-1)*0.3 + index(n-2)*0.2
where "n" is a discrete time position (like year, month, day, whatever). So we are saying that our "average" stock market index is the weighted average of the last three indexes. The weights are [0.5, 0.3, 0,2].
If we trace a graph of these averages over a time interval, we can say that this graph is the convolution of two sequences: the stock index history itself (which is "infinitely" long) versus the list of weights (which has length=3). Of course, drawing an average graph takes a lot of multiplications -- 3 per point, so it is quite intensive.
Moving averages are useful to "smooth out" a curve, avoiding the sharp oscilations of the original signal. In the case of stock market, I may want to devise a "trend line", which is easier to see in a moving average curve than in the sizzly instantaneous market curve. Moving averages make natural lowpass filters; and that's exactly what we are going to do here.
On top of that, there is a very powerful relationship between time-domain and frequency-domain regarding convolutions. Converting a signal from time domain to frequency domain has the power to "convert" a convolution to a simple multiplication, and vice-versa. This means that, if we can define a filter in frequency domain in terms of a multiplication, we can define the same filter in terms of a convolution, which happens in time domain.
Looking at DFT filter code, we have a section where we define a frequency "mask", which we multiply with actual song's spectral distribution, in order to make a bandpass filter:
mask = []
for f in range(0, FFT_LENGTH / 2 + 1):
rampdown = 1.0
if f > LOWPASS:
rampdown = 0.0
elif f < HIGHPASS:
rampdown = 0.0
mask.append(rampdown)
So, in principle, if we convert this frequency mask to a WAV file, and convolve this with our song, we achieve the same filtering result (save for rounding and quantization errors).
This WAV file generated from the filtering mask is NOT an audible wave, it is called the "impulse response". We write it to a WAV file just to demonstrate that we don't need FFT anymore to do filtering. The impulse response of a 3000Hz low-pass filter is something like this:
All we have to do is to convolve an actual audio with this impulse response, and the filtering will happen. That's the code:
#!/usr/bin/env python
import wave, struct, numpy
SAMPLE_RATE = 44100
original = wave.open("NubiaCantaDalva.wav", "r")
filter = wave.open("fir_filter.wav", "r")
filtered = wave.open("NubiaFilteredFIR.wav", "w")
filtered.setnchannels(1)
filtered.setsampwidth(2)
filtered.setframerate(SAMPLE_RATE)
n = original.getnframes()
original = struct.unpack('%dh' % n, original.readframes(n))
original = [s / 2.0**15 for s in original]
n = filter.getnframes()
filter = struct.unpack('%di' % n, filter.readframes(n))
filter = [s / 2.0**31 for s in filter]
'''
result = []
# original content is prepended with zeros to simplify convolution logic
original = [0 for i in range(0, len(filter)-1)] + original
for pos in range(len(filter)-1, len(original)):
# convolution of original * filter
# i.e. a weighted average of len(filter) past samples
sample = 0
for cpos in range(0, len(filter)):
sample += original[pos - cpos] * filter[cpos]
result.append(sample)
'''
result = numpy.convolve(original, filter)
result = [ sample * 2.0**15 for sample in result ]
filtered.writeframes(struct.pack('%dh' % len(result), *result))
Note that there is a section of the program which is commented out. It implements convolution "manually" using two loops and basic arithmetic. This works perfectly well, and I left it (commented) in code just to make clear that convolution is matematically simple. But it is terribly slow; using numpy.convolve() is much, much faster. Actually, this FIR filter is faster than FFT version.
Of course, the FIR filter is fast but we need the impulse response calculated in advance. The following code generates this, and it needs to be run only once (or when you want to change the cut-off frequency of the filter):
#!/usr/bin/env python
LOWPASS = 3000 # Hz
SAMPLE_RATE = 44100 # Hz
import wave, struct, math
from numpy import fft
FFT_LENGTH = 512
NYQUIST_RATE = SAMPLE_RATE / 2.0
LOWPASS /= (NYQUIST_RATE / (FFT_LENGTH / 2.0))
# Builds filter mask. Note that this sharp-cut filter is BAD
mask = []
negatives = []
l = FFT_LENGTH / 2
for f in range(0, l+1):
rampdown = 1.0
if f > LOWPASS:
rampdown = 0.0
mask.append(rampdown)
if f > 0 and f < l:
negatives.append(rampdown)
negatives.reverse()
mask = mask + negatives
fir = wave.open("fir_filter.wav", "w")
fir.setnchannels(1)
fir.setsampwidth(4)
fir.setframerate(SAMPLE_RATE)
# Convert filter from frequency domain to time domain
impulse_response = fft.ifft(mask).real.tolist()
# swap left and right sides
left = impulse_response[0:FFT_LENGTH / 2]
right = impulse_response[FFT_LENGTH / 2:]
impulse_response = right + left
# write in a normal WAV file
impulse_response = [ sample * 2**31 for sample in impulse_response ]
fir.writeframes(struct.pack('%di' % len(impulse_response),
*impulse_response))
Despite the simplicity, a FIR filter is not always faster than a FFT filter. The break-even is around 64 points; more than this, and FFT cheaper. Of course, implementations may change the break-even point: my 512-point FIR filter written in Python is still much faster than the FFT version, so it pays off to use FIR, in particular if my filter must operate in real-time.
The FIR filter is less powerful, in the sense that it can not do every trick that FFT can. If I do some kind of audio morphing in frequency-domain that can not be expressed in terms of a multiplication, it can not be converted to a time-domain convolution, and so it is not realizable as a FIR filter. FIR is a simple tool for a narrow range of jobs, while FFT is a Swiss-army knife, so it is coward to compare them this way.
January 17, 2010 18:47
January 16, 2010
Ok, in my last post I had promised a digital filter, so we do not depend on Audacity filters anymore.
AFAIK there are three main ways to design a digital filter, like a lowpass filter that we needed in AM modulation. The most common is a FIR filter. Another way is an IIR filter. And a third way is using Discrete Fourier Transform (DFT). In this post, I will use the easiest way, which is DFT. Some post in the future will show a FIR version.
DFT can be implemented by a fast algorithm called FFT (Fast Fourier Transform), and most mathematical libraries have a FFT implementation. Python-Numeric is no exception.
DFT converts a time-domain signal, like an audio wave, to a frequency-domain representation. For example, the "spectrum analyzer" feature found in most PC MP3 players and some mini-systems is (or could be) a direct result of DFT calculation:
Apart from showing this fancy spectrum analyzer, DFT has countless other applications.
DFT is not an one-way signal processing feature. Given a frequency domain representation, we can calculate the IDFT (Inverse DFT), to get back the time-domain wave. So, we can get some audio, transform into DFT, manipulate the frequencies (e.g. increasing bass, decreasing midrange, those equalizer things) and then convert back into time-domain.
In our case, we want to build a lowpass/highpass filter, so we just have to zero out the frequencies we don't want, in frequency domain. Such sharp cutting is NOT a good filter design; it causes distortions. But it is ok for a simple demonstration.
Converting to/from frequency domain seems too much work, and in fact there are ways to do filtering without this. FIR filters operate directly on time-domain samples, so they are normally the choice for practical filter applications. But they are not so easy to understand, so this will have to wait for a next post.
For this post, I used a female singer's voice that I like much: Núbia Lafayette. Another reason I used this particular sample is that I own the CD, so nobody can complain about piracy etc. Let's see the original audio:
NubiaCantaDalva.wav
As usual, if your browser does not understand EMBED tags, you can get the file directly from http://epx.com.br/wav.
Now, the filtered version (1000-4000Hz bandpass filter), which sounds like a very small radio receiver:
NubiaFiltered.wav
And the code which does the trick:
#!/usr/bin/env python
LOWPASS = 4000 # Hz
HIGHPASS = 1000 # Hz
SAMPLE_RATE = 44100 # Hz
import wave, struct, math
from numpy import fft
FFT_LENGTH = 2048
# The higher the better, but this seems enough
OVERLAP = 512
FFT_SAMPLE = FFT_LENGTH - OVERLAP
# Nyquist rate = bandwidth available for a given sample rate
NYQUIST_RATE = SAMPLE_RATE / 2.0
# Convert frequencies from Hz to our digital sampling units
LOWPASS /= (NYQUIST_RATE / (FFT_LENGTH / 2.0))
HIGHPASS /= (NYQUIST_RATE / (FFT_LENGTH / 2.0))
zeros = [ 0 for x in range(0, OVERLAP) ]
# Builds filter mask. Note that this filter is BAD, a
# good filter must have a smooth curve, respecting a
# dB/octave attenuation ramp!
mask = []
for f in range(0, FFT_LENGTH / 2 + 1):
rampdown = 1.0
if f > LOWPASS:
rampdown = 0.0
elif f < HIGHPASS:
rampdown = 0.0
mask.append(rampdown)
def bound(sample):
# takes care of "burnt" samples, due some mistake in filter design
if sample > 1.0:
print "!",
sample = 1.0
elif sample < -1.0:
print "!",
sample = -1.0
return sample
original = wave.open("NubiaCantaDalva.wav", "r")
filtered = wave.open("NubiaFiltered.wav", "w")
filtered.setnchannels(1)
filtered.setsampwidth(2)
filtered.setframerate(SAMPLE_RATE)
n = original.getnframes()
original = struct.unpack('%dh' % n, original.readframes(n))
# scale from 16-bit signed WAV to float
original = [s / 32768.0 for s in original]
saved_td = zeros
for pos in range(0, len(original), FFT_SAMPLE):
time_sample = original[pos : pos + FFT_LENGTH]
# convert frame to frequency domain representation
frequency_domain = fft.fft(time_sample, FFT_LENGTH)
l = len(frequency_domain)
# mask positive frequencies (f[0] is DC component)
for f in range(0, l/2+1):
frequency_domain[f] *= mask[f]
# mask negative frequencies
# http://stackoverflow.com/questions/933088/clipping-fft-matrix
for f in range(l-1, l/2, -1):
cf = l - f
frequency_domain[f] *= mask[cf]
# convert frame back to time domain
time_domain = fft.ifft(frequency_domain)
# modified overlap-add logic: previously saved samples
# prevail in the beginning, and they are ramped down
# in favor of this frame's samples, to avoid 'clicks'
# in either end of frame.
for i in range(0, OVERLAP):
time_domain[i] *= (i + 0.0) / OVERLAP
time_domain[i] += saved_td[i] * (1.0 - (i + 0.00) / OVERLAP)
# reserve last samples for the next frame
saved_td = time_domain[FFT_SAMPLE:]
# do not write reserved samples right now
time_domain = time_domain[:FFT_SAMPLE]
# scale back into WAV 16-bit and write
time_domain = [ bound(sample) * 32767 for sample in time_domain ]
filtered.writeframes(struct.pack('%dh' % len(time_domain),
*time_domain))
There is one twist in this code that is worth mentioning. Since any filtering you do in frequency domain will cause phase distortion, you can't just piggyback the processed audio frames one after another, because the audio wave will not transition smoothly from one frame to the next. It will have a different phase at the next frame, so there will be a sharp discontinuity, very annoying to the ears. (If you don't believe me, you can just set OVERLAP to zero and see what happens.)
Both textbook solutions for this (overlap-save and overlap-add) seem not to solve this problem. I have modified the overlap-add method, which apparently solved the issue. My solution fades in the current frame while fading out the end of previous frame. Maybe there is a better solution for this, but I didn't find any.
Another thing that must be remembered by anyone that wants to play with FFT, is that frequency representation is linear. We are used to logarithmic spectrum analyzers, which match the octaves in music, but FFT is different. This means that frequency resolution of a FFT-generated array is much more precise at low octaves than at high octaves.
FFT generates two frequency "lobes": positive and negative frequencies. This sounds very strange, because there can't be negative frequencies in physical world, but DFT has this mathematical feature. In our filter, we had two choices: just zero out negative frequencies (which reduces the final audio volume by half), or filter them in the same fashion as positive frequencies, as we do.
The array generated by FFT consists of complex numbers. This sounds scary but it has a valid purpose: a complex number can represent amplitude AND phase in a single shot. For example, if we find 1+0j in the 3000Hz 'bar', it means that audio has a 3000Hz component with amplitiude 1.0 and in phase with cos(x). If the sample were 0.707+0.707j, it means that the 3000Hz component still has amplitude = 1 (the modulus) but it is 45 degrees ahead of cos(x).
Some applications, for example the spectrum analyzer, do not care about signal phase, so they could employ DCT (discrete cosine transform) whose output is an array of real numbers. The original waveform can be obtained by IDCT, but of course the reconstructed signal will be phase-distorted.
January 16, 2010 16:24
January 14, 2010
Se tudo sair como planejado (e isso é raro) entro em férias em pouco mais de duas semanas, pela primeira vez desde… caramba, desde 2001. É, eu sou louco mesmo, mas isso não vem ao caso (será?).
O que importa é que preciso de um projeto para me manter ocupado durante este período. Já tentei “projetos de verão” antes, mas a maioria foi por água abaixo por falta de tempo, e pela primeira vez este fator não vai estar contra mim. Então decidi tocar uma idéia que tenho na cabeça há MUITO tempo: montar minha própria máquina de arcade (ou, como chamavam na minha terra, “fliperama”). Não, não é essa da foto.
Cresci jogando estas máquinas. Meu jogo favorito até hoje é New Rally-X, mas nunca vou me esquecer da sensação de entrar num arcade e ouvir os sons de Galaga e Choplifter, torcer pelos amigos que jogavam Yie Ar Kung-Fu e me embasbacar com um gabinete “sit down” de Out Run. Ah, os bons tempos… qualquer PC furreba hoje em dia emula estes jogos com um pé nas costas (até meu Dingoo faz isto), mas a forma física da máquina é como um ícone daquela era. É uma forma de me conectar às sensações de um tempo que não volta mais (tá, vou parar, isso tá ficando filosófico demais).
Bom, mão na massa então. O primeiro passo é definir um objetivo (máquina de arcade) e como alcançá-lo (emulando os originais em um PC). O segundo é especificar o hardware do PC. Não é difícil: como disse, qualquer PC moderno emula os clássicos e mais alguns consoles domésticos de brinde, então não é preciso um micro super-poderoso. Acabei optando por uma solução de baixo custo e desempenho adequado.
A placa-mãe é uma IPXLP-MB da Gigabyte/PCWare, com um processador Intel Atom 330 dual-core rodando a 1.6 GHz. É uma placa-mãe completamente integrada, com vídeo, som, rede e tudo o mais on-board, e custou cerca de R$ 300. A ela juntei 1 GB de RAM DDR2, um HD SATA de 250 GB e uma fonte ATX genérica de 250 W reais.
O total ficou em um pouco menos de R$ 600. Poderia ter encontrado mais barato batendo perna na Sta. Ifigênia, mas preferi fazer tudo online, do conforto de minha cadeira, e usar a manhã de sábado que gastaria de loja em loja testando os componentes.
Eu exagerei na fonte (mas não achei uma ATX de potência menor) e no tamanho do HD (mas um de 160 GB é mais caro): um arcade doméstico pode facilmente rodar a partir de um cartão Compact Flash de 16 GB. Se você tiver um HD de 40 ou 80 GB sobrando no fundo do armário, melhor ainda (free parts!). Mas espaço em disco nunca é demais.
Ainda não achei um gabinete mini ITX com preço decente para abrigar tudo isso. Teoricamente eu não preciso de um gabinete, já que vou fazer um. Mas eu realmente gostaria de ter um lugar quentinho e seguro para abrigar a placa até lá. Posso perfeitamente montá-la em uma peça de MDF para os testes, mas tenho criança e gatos em casa, e ambos são curiosos. Melhor evitar acidentes.
Próximo passo: montar os componentes, definir um sistema operacional e fazer testes do software: emulador (MAME), frontend e os jogos em si para ver como é o desempenho. Aqui fica a dúvida: Windows ou Linux? O natural seria usar o Linux, mas há um frontend fantástico para o Windows chamado Hyperspin que me deixou de queixo caído. Infelizmente o desempenho dele em um Atom 270 single-core de 1.6 GHz deixou a desejar. Vamos ver como fica com a placa nova.
Não percam nos próximos posts a conclusão desta emocionante saga. Será que o Rigues vai conseguir, uma vez na vida, concluir um projeto? Ou vai desistir no meio do caminho e transformar o Atom em um servidor de arquivos pra casa? Respostas em breve.
January 14, 2010 04:25
January 13, 2010
A quem interessar possa, o SpeedStream 4200 funciona melhor aqui, com menos perda de pacotes, apesar do SpeedTouch reportar um SNR mais alto. Além disso a tela de diagnósticos do SpeedStream é bem melhor.
Considerando que paguei 15 reais nesse Thomson, também não dá pra reclamar muito :)
January 13, 2010 17:27
January 12, 2010
Radio is the one thing I'd like to play with, but I have not. For several reasons, this interest of mine always end up buried deep at in-tray, and perhaps I'll never fiddle with it, after all.
For a signal to travel over radio waves, it must be modulated first, which means having it transposed to a frequency bandwidth and format suitable to the transmission medium. Modulation techniques are closely related to telecommunications and information theory, and it is possible to play with modulation within the limits of a computer. This is a theme that I must confess I don't understand much, but I am mesmerized by it nevertheless. And to be honest, I find incredible how a couple simple calculations can make the miracle.
In this post, I am going to illustrate the AM modulation techniques, using WAV audio files. To begin with, we need a baseband signal, that is, some audio that fills all available bandwidth from 0Hz. I have recorded my own, ugly voice in order to torture your ears:
(paralelepipedo.wav)
By the way, I am using the EMBED tag to put all audios on page. If it does not work in your browser, you can get the audios at http://epx.com.br/wav.
AM modulation demands that signal is restricted to a bandwidth, so the modulated signal will also "fit" in a predetermined frequency band. So, we need to apply a low-pass filter on my voice, and we define the cut-off at 1000Hz. The result is my voice in lower quality.
(paralelepipedo_lopass.wav)
I have used an Audacity's low-pass filter plug-in. For the next post, I promise to write a filter from scratch, in Python. These Audacity filters that I am using are not perfect, but are enough to illustrate our points.
Amplitude modulation is just multiplying the original signal by a sinusoidal wave, the carrier. In this case, I have chosen a 3000Hz carrier, well above the cut-off of original signal, which is 1000Hz. The carrier noise itself is:
(carrier3000.wav)
Modulation result (voice multiplied by whistle) is:
(amsc.wav)
Note that it is almost impossible to understand the speech. To be exact, this modulation is named AM-SC (Supressed Carrier), for a reason that I will clarify a bit later.
Modulation makes my voice to exchange band from 0-1000 to 2000-4000Hz. Actually, AM modulation creates two "copies" of the original signal, one at 2000-3000Hz band and another at 3000-4000Hz. So, normal AM "wastes" bandwidth because modulated signal occupies twice as much of bandwidth.
Nevertheless, AM is still quite useful, because I can "move" my voice to frequencies suitable for electromagnetic transmission. And, even more important, modulation allows allocation of multiple signals on the same medium, each one occupying a different band, and each one can be recovered separately.
The "normal", old radio AM modulation sounds like this:
(am.wav)
In this modulation, we can hear the carrier, while we could not in AM-SC. The following Python program was employed to generate the modulated AM and AM-SC audios:
#!/usr/bin/env python
import wave, struct, math
baseband = wave.open("paralelepipedo_lopass.wav", "r")
amsc = wave.open("amsc.wav", "w")
am = wave.open("am.wav", "w")
carrier = wave.open("carrier3000.wav", "w")
for f in [am, amsc, carrier]:
f.setnchannels(1)
f.setsampwidth(2)
f.setframerate(44100)
for n in range(0, baseband.getnframes()):
base = struct.unpack('h', baseband.readframes(1))[0] / 32768.0
carrier_sample = math.cos(3000.0 * (n / 44100.0) * math.pi * 2)
signal_am = signal_amsc = base * carrier_sample
signal_am += carrier_sample
signal_am /= 2
amsc.writeframes(struct.pack('h', signal_amsc * 32767))
am.writeframes(struct.pack('h', signal_am * 32767))
carrier.writeframes(struct.pack('h', carrier_sample * 32767))
AM-SC is simply the original signal (whose band has been restricted by a lowpass filter) multiplied by the carrier, which is simply a sinusoidal wave.
AM modulation is the same as AM-SC, except that carrier is added afterwards. This allows the AM receiver to be much simpler, as we will see.
Well, it's time to proof that we can extract the original voice from these modulated audios. The AM-SC modulation is as simple as multiply the signal again by the same carrier, and then passing the result through a lowpass filter. There are the demodulated audios, before and after lowpass:
(demod_amsc_ok.wav)
(demod_amsc_ok_lopass.wav)
As said, AM-SC demodulation is almost equal to modulation. Multiplying a signal twice by the same carrier throws the voice back to its original band (0-1000Hz), and creates a new copy around 6000Hz which needs to be suppresed by the lowpass filter.
This is the AM-SC demodulator code:
modulated = wave.open("amsc.wav", "r")
demod_amsc_ok = wave.open("demod_amsc_ok.wav", "w")
demod_amsc_nok = wave.open("demod_amsc_nok.wav", "w")
demod_amsc_nok2 = wave.open("demod_amsc_nok2.wav", "w")
for f in [demod_amsc_ok, demod_amsc_nok, demod_amsc_nok2]:
f.setnchannels(1)
f.setsampwidth(2)
f.setframerate(44100)
for n in range(0, modulated.getnframes()):
signal = struct.unpack('h', modulated.readframes(1))[0] / 32768.0
carrier = math.cos(3000.0 * (n / 44100.0) * math.pi * 2)
carrier_phased = math.sin(3000.0 * (n / 44100.0) * math.pi * 2)
carrier_freq = math.cos(3100.0 * (n / 44100.0) * math.pi * 2)
base = signal * carrier
base_nok = signal * carrier_phased
base_nok2 = signal * carrier_freq
demod_amsc_ok.writeframes(struct.pack('h', base * 32767))
demod_amsc_nok.writeframes(struct.pack('h', base_nok * 32767))
demod_amsc_nok2.writeframes(struct.pack('h', base_nok2 * 32767))
One little big problem of AM-SC demodulation: the carrier wave generated at receiver must be EXACTLY equal to transmitter, both in frequency and phase. Any difference will wreck the demodulation. In the code above, I simulated two demodulation problems: carrier with wrong phase (90 degrees behind), and with wrong frequency (100Hz higher). The resulting demodulations are very bad, as you can hear below (already lowpassed):
(demod_amsc_nok_lopass.wav)
(demod_amsc_nok2_lopass.wav)
It is very difficult to generate a local carrier at receiver with these constraints. It would be easier in "old radio" AM because the carrier is sent together, so it could be extracted directly from the antenna signal. Due to this, AM-SC is seldom employed in practice. (*)
Now, let's see the "old radio" AM demodulator:
modulated = wave.open("am.wav", "r")
f = demod_am = wave.open("demod_am.wav", "w")
f.setnchannels(1)
f.setsampwidth(2)
f.setframerate(44100)
for n in range(0, modulated.getnframes()):
signal = struct.unpack('h', modulated.readframes(1))[0] / 32768.0
signal = abs(signal)
demod_am.writeframes(struct.pack('h', signal * 32767))
Note how simpler it is, the absence of transcendental functions. AM demodulator is simply a detector, which cuts off the negative part of wave. That's why you can build a galena radio with just a diode and a capacitor, the first being the abs() function and the latter being the lowpass filter.
There are the demodulation results, before and after the lowpass filter:
(demod_am.wav)
(demod_am_lopass.wav)
The remaining noise is likely due to the abs()-based detector in demodulation; In a real-world AM radio, the noise generated by detector would be of a far higher frequency, and lowpass filter would remove it completely. (Remember that we have modulated to a very low frequency, 3000Hz).
Both AM and AM-SC modulations waste bandwidth, because they generate two "copies" of the same content around carrier. Moreover, AM sends the carrier all along with content. We can say that an AM transmitter spends 75% of power just sending "garbage" which does not increase range -- it is there just to help receivers (an heritage from galena days).
But there is a way to improve AM efficiency. Since AM-SC signal has two copies, we can just filter out one of them, sending only the other. This is called AM-SSB (supressed side band). This is how AM-SSB sounds like (lower band removed, only upper band is kept):
(amssb.wav)
Not much different from AM-SC; and in fact the AM-SC demodulator can cope with AM-SSB signal without any changes:
(demod_amssb_ok.wav)
(demod_amssb_ok_lopass.wav)
Besides saving power by sending only one copy of original content, without the carrier, AM-SSB has a second, definite advantage over AM-SC. See what happens when we demodulate AM-SSB with an out-of-phase carrier:
(demod_amssb_nok_lopass.wav)
The voice has been recovered perfectly. So, an AM-SSB receiver does not need to worry with carrier's phase. Only the frequency is important, and it must be watched closely; any frequency deviation causes audio distortion pretty much like AM-SC:
(demod_amssb_nok2_lopass.wav)
AM-SSB is the standard modulation for ham radio, police, aviation, analog TV video etc. And the same technique is employed in FDM multiplexers and countless other devices that need to fit many signals in a single path. Removal of the unneeded sideband in tranmission can be done with an analog filter, but such filter is difficult and expensive to build. DSP-powered radios use a technique called Hilbert Transform to generate the AM-SSB signal directly, without the need of a filter.
(*) A surprising usage of AM-SC is the FM stereo. The second audio channel is encoded
within the audio signal, with a 38KHz carrier. A 19KHz pilot tone is transmitted all the time so the receiver can generate a local carrier with perfect frequency and phase. Of couse this implies that FM audio can not be more treble than 19KHz.
January 12, 2010 22:39
Rádio é uma coisa com a qual eu gostaria de ter mexido, mas ainda não o fiz. Por diversas razões, este interesse acaba sempre em segundo plano frente a coisas mais urgentes, e quem sabe eu nunca brinque com rádio mesmo.
Para um sinal viajar nas ondas de rádio, ele tem de ser modulado, ou seja, transposto para uma freqüência e formato adequados à transmissão. As técnicas de modulação do rádio têm parentesco próximo com informática e telecomunicações, É possível "brincar" com modulação no computador mesmo. É um tema do qual eu confessadamente não entendo muito, mas acho fascinante. E para ser honesto, acho incrível que técnicas aparentemente simples façam o milagre de modular e demodular um sinal.
Neste post, vou ilustrar as técnicas de modulação AM. Quem tiver interesse em aprofundar-se no tema, tente achar o livro "Modem e transmissão de dados", de Fábio de Azevedo Montoro, editora Érica. Apesar de velho, é um dos poucos livros técnicos escritos por brasileiro que presta.
Bem, vamos começar com um sinal de banda-base, ou seja, que ocupa toda a banda disponível a partir de 0Hz. Uma gravação da minha própria voz, pra sacanear com os ouvidos de todo mundo:
(paralelepipedo.wav)
A propósito, estou usando a tag EMBED para colocar o áudio na página. Se não funcionar, você pode pescar os arquivos em http://epx.com.br/wav.
A modulação AM exige que o sinal original ocupe uma banda limitada, de modo que o sinal modulado também "caiba" dentro da banda desejada. Assim, precisamos pegar o sinal original e passar num filtro passa-baixas, até 1000Hz. O resultado é a mesma voz um pouco mais abafada:
(paralelepipedo_lopass.wav)
Para o próximo post, prometo escrever um filtro em Python. Desta vez bateu a preguiça e usei os recursos do Audacity mesmo para fazer a filtragem. Não é perfeito mas é suficiente para nossos fins.
A modulação AM consiste simplesmente em multiplicar o sinal original por uma onda senoidal, a portadora. No caso, escolhi uma portadora de 3000Hz, bem distante da banda ocupada pela minha voz filtrada (1000Hz). O ruído da portadora em si é:
(carrier3000.wav)
O resultado da modulação (voz x apito) é:
(amsc.wav)
Note que a voz e as vogais ficaram agudas e quase irreconhecíveis. A rigor, esta modulação denomina-se AM-SC (AM com portadora suprimida), por razão que logo esclareço.
A modulação faz minha voz mudar de banda: ela passa a ocupar uma faixa de 2000 a 4000Hz. Na verdade, a modulação AM produz duas "cópias" do sinal original: uma ocupando a banda 2000-3000Hz, e outra ocupando a banda 3000-4000Hz. A modulação AM normal "desperdiça" banda pois o sinal modulado ocupa o dobro de espaço que o original.
Mesmo assim, AM é muito útil, pois obviamente eu consigo "mover" a minha voz para qualquer faixa de freqüências que seja conveniente à transmissão. Mais importante que isso: a modulação permite que diversos sinais ocupem o mesmo meio, bastando que cada sinal tenha uma portadora diferente e as bandas não se sobreponham.
Podemos modular em AM "normal":
(am.wav)
Note que o apito da portadora pode ser claramente ouvido. Esta é a modulação utilizada pela rádio AM tradicional.
Segue o programa Python que gerou as modulações AM-SC e AM "tradicional":
#!/usr/bin/env python
import wave, struct, math
baseband = wave.open("paralelepipedo_lopass.wav", "r")
amsc = wave.open("amsc.wav", "w")
am = wave.open("am.wav", "w")
carrier = wave.open("carrier3000.wav", "w")
for f in [am, amsc, carrier]:
f.setnchannels(1)
f.setsampwidth(2)
f.setframerate(44100)
for n in range(0, baseband.getnframes()):
base = struct.unpack('h', baseband.readframes(1))[0] / 32768.0
carrier_sample = math.cos(3000.0 * (n / 44100.0) * math.pi * 2)
signal_am = signal_amsc = base * carrier_sample
signal_am += carrier_sample
signal_am /= 2
amsc.writeframes(struct.pack('h', signal_amsc * 32767))
am.writeframes(struct.pack('h', signal_am * 32767))
carrier.writeframes(struct.pack('h', carrier_sample * 32767))
A modulação AM-SC é simplesmente a multiplicação do sinal original, cuja banda é previamente confinada numa faixa, por uma portadora, que é nada mais que uma onda senoidal.
A modulação AM tradicional é a mesma coisa, exceto que a portadora é adicionada ao sinal modulado, depois da multiplicação. Isto permite que o receptor seja consideravelmente mais simples, conforme veremos.
Bem, agora tá na hora de demodular, ou seja, tentar recuperar o sinal original, usando os arquivos WAV modulados. A demodulação AM-SC consiste em nova multiplicação pela portadora, seguida de um filtro passa-baixas. Segue os resultados, antes e depois do filtro:
(demod_amsc_ok.wav)
(demod_amsc_ok_lopass.wav)
A demodulação AM-SC é essencialmente igual à modulação. O efeito de multiplicar duas vezes pela mesma portadora joga de volta o sinal para sua banda original (0-1000Hz), criando uma nova cópia indesejável em torno de 6000Hz, que removemos com o filtro.
Segue o código do demodulador AM-SC:
modulated = wave.open("amsc.wav", "r")
demod_amsc_ok = wave.open("demod_amsc_ok.wav", "w")
demod_amsc_nok = wave.open("demod_amsc_nok.wav", "w")
demod_amsc_nok2 = wave.open("demod_amsc_nok2.wav", "w")
for f in [demod_amsc_ok, demod_amsc_nok, demod_amsc_nok2]:
f.setnchannels(1)
f.setsampwidth(2)
f.setframerate(44100)
for n in range(0, modulated.getnframes()):
signal = struct.unpack('h', modulated.readframes(1))[0] / 32768.0
carrier = math.cos(3000.0 * (n / 44100.0) * math.pi * 2)
carrier_phased = math.sin(3000.0 * (n / 44100.0) * math.pi * 2)
carrier_freq = math.cos(3100.0 * (n / 44100.0) * math.pi * 2)
base = signal * carrier
base_nok = signal * carrier_phased
base_nok2 = signal * carrier_freq
demod_amsc_ok.writeframes(struct.pack('h', base * 32767))
demod_amsc_nok.writeframes(struct.pack('h', base_nok * 32767))
demod_amsc_nok2.writeframes(struct.pack('h', base_nok2 * 32767))
Um problema da demodulação AM-SC é que a portadora do receptor tem de ser EXATAMENTE igual à do transmissor. Qualquer diferença em fase ou frequência causa distorções graves no sinal. No código acima, procurei simular problemas com moduladora fora de fase (usando seno em vez de cosseno) e com frequência errada (3100Hz em vez de 3000). Os resultados desastrosos podem ser ouvidos abaixo, já filtrados pelo passa-baixas:
(demod_amsc_nok_lopass.wav)
(demod_amsc_nok2_lopass.wav)
Esta necessidade de manter uma portadora precisa no receptor, sem que haja muitos meios de recuperá-la a partir do sinal modulado, acaba tornando o AM-SC uma modulação pouco prática. Um dos poucos usos reais é a trasnsmissão estéro FM, onde o segundo canal é embutido dentro do próprio áudio, com portadora em 38KHz e sinal-piloto transmitido o tempo todo para garantir a perfeita demodulação no receptor.
Vejamos agora o código do demodulador AM tradicional:
modulated = wave.open("am.wav", "r")
f = demod_am = wave.open("demod_am.wav", "w")
f.setnchannels(1)
f.setsampwidth(2)
f.setframerate(44100)
for n in range(0, modulated.getnframes()):
signal = struct.unpack('h', modulated.readframes(1))[0] / 32768.0
signal = abs(signal)
demod_am.writeframes(struct.pack('h', signal * 32767))
Note que este demodulador nem usou seno ou cosseno. Ele é meramente um detector, ou seja, transforma sinais negativos em positivos, o que num rádio real seria feito por um diodo ou ponte de diodos. É isto que permite um receptor AM galena funcionar: basta um diodo e um capacitor.
Segue o resultado antes e depois do filtro passa-baixas:
(demod_am.wav)
(demod_am_lopass.wav)
Ficou um ruidozinho de portadora mesmo na versão filtrada, mas isto se deve muito à filtragem ruim (seria mais fácil filtrar se a freqüencia da portadora fosse muito mais alta que a do sinal original, como é o caso da transmissão de rádio).
Tanto a modulação AM-SC quanto AM tem a séria desvantagem de desperdiçar banda, pois geram duas "cópias" do mesmo sinal ao redor da portadora. Além disso, AM transmite a portadora o tempo todo junto com as cópias. No fim das contas, um transmissor AM gasta 75% da energia transmitindo "bobagens", que não aumentam seu alcance, servem apenas para facilitar o trabalho do receptor.
Mas existe uma forma de resolver isto. Já que o sinal AM-SC possui duas cópias, podemos filtrar uma fora, transmitindo apenas a outra. Isto denomina-se AM-SSB (AM com banda suprimida). Segue o sinal AM-SSB com a cópia inferior (2000-3000Hz) filtrada:
(amssb.wav)
Não parece muito diferente do sinal AM-SC, e de fato o mesmo demodulador AM-SC consegue recuperar a voz original a partir do sinal AM-SSB:
(demod_amssb_ok.wav)
(demod_amssb_ok_lopass.wav)
Além de não desperdiçar nenhuma energia (apenas uma cópia do sinal é transmitida, sem a portadora), o AM-SSB tem uma outra vantagem, ainda mais surpreendente. Veja o que acontece com a voz demodulada com uma portadora fora de fase:
(demod_amssb_nok_lopass.wav)
A voz foi recuperada normalmente. Desta forma, o receptor AM-SSB não precisa se preocupar com a fase da portadora gerada localmente, o que facilita muito as coisas. A freqüência ainda é importannte, qualquer desvio causa a mesma distorção que a observada em AM-SC:
(demod_amssb_nok2_lopass.wav)
AM-SSB é a modulação padrão utilizada pela radiocomunicação em geral (radioamador, polícia, aviação, etc. etc.). Além disso, a mesma técnica básica é utilizada em multiplexadores FDM, no sinal de vídeo da TV analógica, e na transmissão do canal estéreo das rádios FM (o segundo canal é modulado dentro do áudio, com portadora de 38KHz e largura de 19KHz).
A remoção da cópia excedente no transmissor pode ser feita com um filtro analógico, embora seja difícil e caro construir o filtro com as características necessárias. Rádios com processamento digital de sinais podem utilizar uma técnica denominada Transformada de Hilbert, o que dispensa este componente.
January 12, 2010 18:29
January 11, 2010
De vez em quando a gente se dá uns presentinhos, certo? Pois hoje foi meu dia.
Na verdade eu já vinha prometendo há um bom tempo que ia dar um laptop a minha esposa. Hoje resolvi aproveitar as liquidações de pós-festas e comprei um netbook Lenovo S10. Como comprei em promoção, peguei a versão 1 mesmo. Eu já havia feito alguns comparativos antes, e não vi grandes diferenças do 1 pro 2, então a diferença de 30% no preço foi o ponto chave na decisão pela compra. Até agora parece que vai bem. Não é láááá muito veloz, mas é o suficiente pra acessar a internet confortávelmente, e é bem leve. Muito prático.
Pra mim (e pra ela também) comprei uma TV de LED 40″ da Samsung. Pelo que entendi, essa é a entry-level das LED. Fiquei na dúvida entre uma LCD da Sony ou essa LED da Samsung, mas várias coisas pesaram: a diferença de preço (a Samsung estava com 35% de desconto), a taxa de contraste, design e consumo de energia. Eu sei que alguém vai me crucificar e dizer que a Sony teria sido uma compra melhor, mas além dos quesitos técnicos existem outras questões subjetivas que vão de cada um na hora de fazer uma compra, portanto…
Bom, pra quem está passando por aqui fica uma dica com relação a essa TV: o som é pééééssimo. Por sorte eu tenho uma caixinha de som com subwoofer que resolve o meu problema. Não sei se eu ia conseguir viver com aquele som de radinho de pilha.
Com relação à qualidade da imagem, bom, como falei antes, é muito subjetivo. Cada um tem seu gosto. Eu gostei muito da qualidade (não é exatamente o que eu queria, mas pra uma entry-level já está muito bom). A NET ficou a mesma coisa (óbvio, ainda não tem HD), os meus vídeos estão excelentes, e o Playstation está absurdamente bom. Só ainda não testei o receptor DTV (é, tem). Bem, o fato é que comparado com a minha antiga LG de 32″ a qualidade é de outro mundo. A LG fazia 1080i, enquanto a Samsung (obviamente) faz 1080p, e ainda tem os 120Hz. Só do “i” pro “p” não deveria fazer absurda diferença, mas o fato é que eu comprei a LG na época em que LCD ainda era novidade, assim como estou comprando a LED quando a coisa ainda está meio nova. Ou seja, tenho certeza de daqui a uns 2 ou 3 anos essa Samsung vai pro quarto, e a LG pra fora de casa.
Falando em quarto, aproveito pra me despedir da minha TV de CRT (tubo) Sony Trinitron de 21″. Essa TV sobreviveu 12 longos anos, e 3 mudanças de estado (além de várias mudanças dentro de uma mesma cidade). A TV é guerreira, e merece uma menção honrosa. Pena que vou ter que vender, só por que não tenho espaço pra ficar com ela aqui em casa (fiquei com a LED na sala, e a LCD no quarto). Quem quiser negociar, manda uma mensagem (aê, merchan!).
Ah, outra coisa que me livrei foi o switch HDMI. A LG só tinha 1 entrada, e como eu tinha 3 dispositivos, precisei comprar esse “aparato”. Uma coisa a menos na estante.
É isso aí. O que era bom, ficou melhor. Agora, se vira nos 30 pra pagar tudo isso! Hahaha! 
Tagged: hardware, hdmi, novidades, playstation, tv
January 11, 2010 00:52
January 10, 2010
Despedidas
Vou durmir e despois despertar do outro lado, do outro lado do mar
Vou fechar os meus ollos e estar do outro lado, do outro lado do mar
Vou escoitar despedidas que quedan prendidas nos brazos do vento
Vou lembrar mil historias de amor sumerxidas no fondo do mar
Vou durmir e despois despertar do outro lado, do outro lado do mar…
Três anos atrás, em uma viagem memorável, visitei a Galícia. Um dos pontos altos do evento foi um show de grupo de música galega Da Outra Margem. As músicas eram lindas, divertidas, o show foi fantástico. Porém uma música me marcou muito, esta acima. Conversei com o cantor e autor da música, Luis Lechuga, que algum tempo depois me enviou o mp3. Esta música fala muito para quem como eu tinha um oceano de distância com a pessoa amada. E é engraçado que muitas vezes pensei em blogar esta música em uma das viagens sobre este oceano, mas algo acontecia e eu sempre esquecia. E curiosamente, esta vai ser a última viagem sobre este oceano que fazemos separados…
Posted in viagens
January 10, 2010 14:45
January 09, 2010
Não me mande embora não sou sua amante
Sou sua mulher e venho lhe buscar
Você vai comigo por bem ou por mal
Ou pego essas quengas e quebro no pau
Eu rodo a baiana mas levo você
Sou em quem trabalho e pago sua bebida
Sua roupa, seu carro e sua comida
Por isso eu quero e tenho direito
De ter você homem safado pra mim
-- banda Companhia do Calypso, música "Homem Safado"
Se um homem disser publicamente que "pago as contas logo tenho direitos", vai para a cadeia, não?
January 09, 2010 17:51
January 08, 2010
Faz pouco mais de 1 semana que larguei de vez o consumo da Coca-Cola em casa, e dessa vez tomei o cuidado de substituir por outras bebidas que também incluem cafeína, especialmente o chá preto. Mantive também o mesmo consumo de café de antes.
Mesmo assim, aconteceu o de sempre: eu, que não precisava de despertador e sempre acordava com qualquer barulhinho, agora durmo a manhã inteira, se não houver nenhuma intervenção impactante.
Como eu já dormia bem e em número de horas suficiente, vou torcer pra adaptação acontecer logo, senão vou ter de reativar o despertador, e aí vai ser de fato uma perda e uma pena.
January 08, 2010 14:12
Tenho uma confissão a fazer: durante a cobertura da MacWorld e CES 2009 eu fiz nada menos que 72 vídeos, e só uma dúzia deles chegou a ir ao ar. É fácil entender o motivo: a correria é imensa, banda larga boa o suficiente para upload de vídeos está, por incrível que pareça, disponível apenas na sala de imprensa e há coisas demais para uma pessoa só administrar. O resultado é que acabei subindo apenas o que considerei “mais interessante” e deixei o resto “para depois”. E o depois acabou se arrastando por um ano.
Mas ontem, vendo as fotos do povo na fila para a apresentação do Ballmer, me lembrei dos vídeos e corri atrás. Encontrei vários clipes de bastidores (como a fila para o Ballmer, igualzinha), produtos curiosos e cenas inusitadas que encontrei por lá, e decidi começar a postar tudo isso no YouTube. Se eu for esperar até tomar vergonha na cara para editar e organizar tudo como se deve os clipes nunca irão ao ar. Portanto, estou postando o material “cru” ou com edição mínima.
A imagem não é das melhores (eu tinha acabado de comprar a filmadora em uma Best Buy dois dias antes, e estava aprendendo a usar), as cenas tremem (tente manter uma câmera estável depois de dormir só quatro horas na noite anterior e tomar cinco doses de espresso para compensar) e o áudio estoura, mas mesmo assim eles ainda valem a pena. São uma visão curiosa dos “bastidores da reportagem”, pra quem se interessa por como as coisas funcionam ou acha que a cobertura de uma CES é um paraíso de uma semana em cassinos, comida farta e gadgets legais.
Começo por um review da… Mana Potion, que não tem muito a ver com a CES propriamente dita: Mana Potions são energéticos vendidos para gamers nos EUA, batizados como as poções geralmente usadas para recuperar energia “espiritual” ou mágica em jogos de RPG. Tem uma composição diferente de bebidas como Red Bull, e prometem te deixar alerta sem “danos colaterais” como a agitação excessiva ou cansaço insuperável quando o efeito acaba.
Comprei (a US$ 3.50 o frasco) como última tentativa de encontrar uma alternativa às cinco doses de espresso. Funciona? NÃO:
O efeito colateral mencionado no vídeo aconteceu mesmo, e não foi NADA agradável. Câmera e “trilha sonora” por André Faure. Mais clipes em breve.
January 08, 2010 03:01
January 07, 2010
January 06, 2010
Chuva de verão chegando agora a Jurerê após uma tarde abafada.
January 06, 2010 19:15
Since I first heard about the filming of the Lord of the Rings movies, my life was almost divided into two stages: before I heard about it, and after that. The Lord of the Rings was the first book I read (at age 4 or 5, I don’t remember exactly), and I read it at least 50 times since that. So the waiting that movie was really, really expected.
After the last of the LOTR movies, I was a bit lost. There was no more need to wait for the next movies. I felt almost like ‘the cinema has got to its top, and I don’t think any other movie will entertain me, and allow me to go to the other worlds as the LOTR did’. For several years, I was waiting for the next LOTR movies like a child who is waiting for the Santa’s gifts at the end of the year. And suddenly I felt like all the wishes were fulfilled.
However, some new movies appeared and made me feel almost as when I was waiting for the Fellowship of the Ring, The Two Towers and The Return of the King – movies like the Spiderman series, The Jason Bourne series, the last episode in the Starwars movies (well, the 3rd episode to be correct), Pirates of the Caribbean, 300, the new Batman and The Dark Knight movies, and many others. However, most of those movies appeared before 2009, so I started the year thinking ‘oh my.. this year will be a really boring one, with nothing interesting to watch’.
Well, to sum it in a few words.. I was completely wrong. The year of 2009 brought me a lot more excellent movies that I could possible hope. Just to name a few: Watchmen (until the very end of the year, I thought it would be simple the best movie in a long long time), 500 Days of Summer, Zombieland, Terminator Salvation, Inglorious Bastards, Taken, Star Trek, Pandorum (almost-the-best SciFi movie in a long long time), District 9, and many many others. I have to say that all those movies were great, and left a mark in my memories.
And then Avatar came and put them all into the background.
I already expected a lot from this movie, and also was quite afraid that it would not live up to the expectations. However, I had a lot of trust in James Cameron – the cinema magician who created Terminator, Aliens, True Lies, Abyss and Titanic.. and he made yet another cinema miracle. In my opinion, Avatar is the biggest happening in the cinema world of this decade. And it is specially true for the ones out there who happen live some part of their life in different worlds – worlds created by writers, cinema guys, or RPG games.
The writers manage to create great worlds, and make you believe that they are real in your mind. Role-playing games put you in those worlds, and make you part of it. But with Avatar, James Cameron was the first guy to in fact create a different world, and let us to live in it for a while. That world feels so real, that you forget about all the computer-generated graphics, effects, and a (few) plot holes, so you just lose a few hours of your real life to spend them on Pandora.
So, to sum it all up – if you haven’t done it yet, go watch Avatar – and, if possible, watch it in 3D. You’ll probably feel the same feeling as the people in the beginning of the past century had, by watching the first movies ever.
I think that resumes pretty much everything I have to say about it .
January 06, 2010 00:13
January 05, 2010
2009 foi-se. Cumpri satisfatoriamente minhas metas do ano, por incrível que pareça:
- Acabar com a barriga: Na última semana do ano eu estava nadando pelo menos 6km por semana e caminhando 10km. Com isso, fiquei tão magra que todo mundo reclamou dos meus 42.5kg. Nada que eu não tivesse recuperado com rabanadas e panetones e dia 4 de janeiro lá estava eu reclamando dos quilos ganhos no ano novo. Mas nada que meu ritmo de caminhadas e natação não resolva. Só de ontem pra hoje foram mais 2km de natação e 5km de caminhada. Quem me viu, quem me vê.
- Peixetarianismo: Carne vermelha não me faz falta e só como peixes nos fins de semana. Cheguei inclusive a passar um mês vegetariana mesmo, mas foi um certo exagero. Troquei o leite de vaca por soja porque, cara, eu bebo MUITO leite. Mas descobri que curto culinária vegetariana bem feita e comida saudável é gostosa – eu sempre detestei comida gordurosa. E, vocês sabem, macarrão com molho de tomate é ótimo e é ovo-lacto-vegetariano. Com MUITO molho de tomate.
- Comer lagosta: Passei uma semana no nordeste, deve ser o melhor lugar do mundo para se comer. Eu me alimentei quase que exclusivamente de camarão e lagosta, e paga-se MUITO POUCO por INCRÍVEIS camarões e lagostas lá. Logo, não apenas comi lagosta, como COMI MUITA LAGOSTA. Lagosta é sensacional, de qualquer jeito. O melhor prato foi na tal barraquinha de pescador: por 110 reais, 5 lagostas grandes, uma pilha enorme de camarões de bom tamanho, muito peixe, pirão e acompanhamentos. Serviu 4 pessoas esfomeadas e sobrou. Tô dizendo que o nordeste é o paraíso. E eu posso viver sem carne vermelha.
- Viajar mais: Ok que eu não saí do país, mas fui para Natal, passei uma semana num resort na beira do mar, comi, tomei sol, fui feliz. Foi bom o bastante.
- Fiquei sem meta cinco, não inventei nada bom o suficiente para uma meta cinco. Mas, acho que foi um bom ano.
Em breve postarei metas para 2010. Tenho tanta coisa para fazer que preciso estabelecer metas realísticas, ou terminarei o ano frustrada.
January 05, 2010 19:02
January 04, 2010
## Ou como queimar o filme em um post, ou ainda mais “esse post é para dar risada de um otário” ##
Eu ainda vou fazer um filme sobre isso. Sobre um cara que passa bom tempo da vida sozinho, tem as experiências românticas mais losers do universo, e quando tenta reclamar para alguém do que está acontecendo a única resposta inútil que eu recebo é: “ah, tu vai achar alguém”.
Pois eu confessso. Não tenho medo de morrer, mas que seja rápido, que pelo menos eu não sinta muita dor, mas um dos únicos medos que tenho na minha vida é morrer sozinho. É bom ter amigos, mas as coisas não chegam a um certo nível quando se tem apenas amigos, e não uma namorada. As vezes ainda sou obrigado a conviver com perguntas sobre o meu casamento ou a namorada. Isso ajuda muito, claro, a f* o meu bom humor e aumentar um pouquinho daquela tristeza do ser sozinho que eu tenho.
E eu fico mais triste porque as coisas que acontecem são simplesmente as mais impossíveis. Não tenho inveja dos meus amigos, mas acabo perdendo a companhia sempre aos 45 minutos do segundo tempo. E é sempre assim. Eu posso estar conversando com ela a 3 horas sem parar, e em 1 minuto chega alguém e fica com a guria na minha frente. Na minha cidade, isso se chama padeiro, e sim, eu sou um grande padeiro, prazer!
E tudo, e tudo é sempre impossível pra mim. Sabe aquela cantada ridícula? Funciona com todos os meus amigos. Sabe aquela coisa de tratar bem a mulher? Sim, eu faço, mas sou o maior otário do mundo, porque alguns minutos depois vai chegar um cara dando um soco na cara da guria e ela vai ter um orgasmo imediato e vai ficar com o cara na minha frente.
E não, as coisas não estão bem por esse lado. No Brasil, eu morria trabalhando e não perdia tempo com essas coisas, e deixava para lá, por mais que aquele sentimento estivesse escondido lá no fundo, eu conseguia esquecer. Agora faz 15 dias que eu estou na Alemanha sem trabalho ou algo que ocupe realmente a mente e a época do amor explode para os alemães. Todo mundo casado, namoradas lindas, e é claro, tem aquela mulher linda que eu já nem perco muito tempo para não sofrer. É foda ter essas emoções perturbadoras explodindo e não saber lidar com elas. Eu até pouco tempo eu empurrava toda a história com a barriga, mas agora não dá mais.
Ou seja, vou ter que meditar muito mais, porque preciso ocupar a minha cabeça com alguma outra coisa que não seja relacionamentos, já que não consigo lidar muito bem com isso. Na verdade eu sou simplesmente o otário que tem muito amor para dar e coisas bem bacanas, mas fodo tudo na apresentação.
Chances boas eu tenho, posso ter um emprego razoável, um apartamento, tudo, realmente tudo o que torna a vida de uma pessoa estável, mas por alguma razão, relacionamentos para mim não funcionam, de maneira alguma. Ou seja, ter que conviver com um grande GAP é realmente f*, já que todo mundo adora exibir o seu amor por ai.
Uma pergunta que não cala. Como acredito em reencarnação, acredito que fiz muita mulher sofrer no passado, como alguém que as machucou muito fisicamente, ou emocionalmente. Até agora é a única razão ou desculpa que me dá um certo consolo, pois não ter respostas concretas e conselhos miojo, de botequim, é foda.
Aos que me conheceram em POA, e acharam que eu era gay, se deram muito mal. Não é a minha escolha, respeito quem a tem, mas era muito divertido tirar sarro com meus amigos. Embora eu seja um loser, e não tenha uma namorada do meu lado, gosto do sexo oposto.
Pois é, se você está de saco cheio da namorada(o), esposa(o), pelo menos considere-se sortudo, pois tem gente que não tem capacidade alguma de ter isso, e tem que conviver com isso durante a vida.
Pronto, desabafei, precisava, pelo menos falar. Ou escrever o que eu sinto.
Esse foi o post mais confessional que eu fiz, portanto os comments estão desabilitados. Have fun! (Espaço para risadas)
January 04, 2010 22:51
From time to time, several questions appear asking about the support for Mandriva products – either with bugfix updates, or the security ones. According to the policy, each update is assigned a specific advisory, which can affect one or more distributions. Besides, each advisory could receive an errata, to correct a regression caused by a previous update. Moreover, most of the updates fall into the bugfix (e.g., fixing some bad behavior or crash or simple improving the application), or security (fixing a security issue which could lead to remote compromising of the system, denial of service or other nasty effects) categories. We also have the general updates category, but this is not that different from bugfix updates, so I’ll count both of them together here.
Now that we are entering the year 2010, I thought that it would be interesting to give you some quick follow-up on how many updates were done during the last few years.
In 2009, there were a total of 436 security updates for all Mandriva-supported packages (e.g., the packages in Main repository), and a total of 288 bugfix updates. The bugfixes are usually provided by the package maintainers, who are responsible for issuing the fix/patch, properly testing the updated package, and send it to the secteam. Secteam does the final validation, signs the package with the update key, and releases the advisory (which is sent to a mailing list and to the Mandriva web site). Later, the packages are sent to the mirrors, and become available to the users.
With security updates, it is a bit different. The entire process is usually handled by the secteam, which is responsible for identifying the security issue, locating the relevant patch or solution, updating and testing the fix, and releasing the updated packages. After that, those updates have a similar fate to the bugfix ones (e.g., signing, releasing the advisory, and so on).
If we look into the detailed numbers, things become quite more interesting:
- In 2006, we had 67 bugfix updates and 250 security updates. In total, 10769 RPM files were provided as updates.
- In 2007, there were 144 bugfix updates and 262 security updates, with a total of 17786 updated rpm files
- In 2008, we had 213 bugfix updates and 264 security updates, totaling 25718 rpms provided as updates
- And in 2009, there were 288 bugfix updates and 436 security updates, with a total of 41024 rpms provided as updates.
So I’d say that Mandriva users are pretty well supported .
January 04, 2010 17:57
January 03, 2010
Como eu disse no post sobre a maria-fumaça, alguns aspectos das ferrovias brasileiras fazem a festa dos modelistas. Um deles é a antiguidade, o outro é o estado de abandono em que estão muitos elementos, como as estações ferroviárias abandonadas. Modelar um objeto velho é mais fácil e divertido que um novo: pode-se cometer mais erros e depois dar um "banho" de giz-pastel em pó para "sujar" tudo. Pode-se até usar o efeito Antique nas fotos para disfarçar ainda mais :)
Assim, depois de uma visita à estação do Rio Natal, decidi fazer uma estação parecida em estireno. Infelizmente eu só pensei nisso depois de voltar; e só tinha feito uma foto da estação, que não era suficiente para extrair as medidas. Assim o negócio foi fazer algo "inspirado" no prédio, em vez de uma cópia fiel. Na próxima vez vou tirar mais fotos e fazer um modelo mais próximo do real.
A construção, como eu disse, foi feita com chapas de estireno: 1,5mm para as paredes, com reforços internos onde há portas, e 0,3mm para telhado e os relevos nas paredes, no perímetro de cada parede e nas janelas, portas etc. Cada parede foi preenchida com uma massinha bastante versátil, chamada "Tapa Tudo", para dar a textura de parede de concreto.
Essa massa é interessante para muitas tarefas domésticas e também de modelismo: pode ser misturada com um pouco de água para ficar mais mole ou até líquida. Sua principal característica positiva é não encolher quando seca. Mesmo quando liquefeita com água ela tende a formar uma espuma e encolhe pouco, embora demore dias para secar neste caso.
Vi tambores de 5 ou de 10 litros de "Tapa Tudo" na loja; imagino que muita gente esteja usando-a em lugar da massa corrida para alguns tipos de acabamento, pois massa corrida encolhe e racha se aplicada em camada pouquinha coisa mais grossa.
Apesar dos inúmeros erros que cometi (foi a primeira coisa que construí em chapas de estireno), o resultado final ficou interessante. Alguns toques internos como piso de madeira (ripas de balsa envernizadas) foram adicionados visando uma futura iluminação interna, já que hoje mal se pode vê-los.
January 03, 2010 15:03
