xwords/xwords4/dawg/remove-dirty.py

#!/usr/bin/python3

# Invoked with path to bad words list as single parameter, and with a
# stream of words via stdin, loads the bad words into a map and for
# every word in stdin echos it to stdout IFF it's not in the map.

import sys

dirtyMap = {}
dirtyList = sys.argv[1]
for f in open(dirtyList):
    dirtyMap[f] = True

for word in sys.stdin:
    if word in dirtyMap:
        sys.stderr.write( sys.argv[0] + ": dropping: " + word )
    else:
        sys.stdout.write( word )
need to specify which python now 2021-11-22 02:36:54 +01:00			`#!/usr/bin/python3`
add ability to filter out "dirty" words If a Makefile defines a dirty word list then a new python script is invoked to filter for and remove those words as the dict is being built. So far I have for English only, which makes sense because only English wordlists are built-in on Android and Google's rating system cares only about what's built in. 2017-05-05 07:45:27 +02:00
			`# Invoked with path to bad words list as single parameter, and with a`
			`# stream of words via stdin, loads the bad words into a map and for`
			`# every word in stdin echos it to stdout IFF it's not in the map.`

			`import sys`

			`dirtyMap = {}`
			`dirtyList = sys.argv[1]`
			`for f in open(dirtyList):`
			`dirtyMap[f] = True`

			`for word in sys.stdin:`
			`if word in dirtyMap:`
			`sys.stderr.write( sys.argv[0] + ": dropping: " + word )`
			`else:`
			`sys.stdout.write( word )`