TypeError: __init__() got an unexpected keyword argument ‘max_len‘
·
Problem Descriptions:
When I implement a Chinese character-based Bert tokenizer, I need first to extend the BertTokenizer model, then initiate the parameters, and finally create a character-based tokenizer method.
The following is the parameter initalization code.
class BERTChineseCharacterTokenizer(BertTokenizer):
def __init__(self, vocab_file, do_lower_case=True):
super(BERTChineseCharacterTokenizer, self).__init__(vocab_file, do_lower_case)
def char_tokenize(self, text, unk_token='[UNK]'):
Errors:
TypeError: __init__() got an unexpected keyword argument 'max_len'
Solution: initialize the max_len value.
class BERTChineseCharacterTokenizer(BertTokenizer):
def __init__(self, vocab_file, do_lower_case=True, max_len=512):
super(BERTChineseCharacterTokenizer, self).__init__(vocab_file, do_lower_case, max_len=512)
def char_tokenize(self, text, unk_token='[UNK]'):
更多推荐


所有评论(0)