Found a Google Translate endpoint that doesn't require an API key.

This isn't a bug report, just a sharing of some findings while looking through minified source code.

While digging through the source code of Google's Google Dictionary Chrome extension, which has support for translating via Google Translate, I found the endpoint they use in order to do just that. Since googletrans frequently runs into 5xx errors, it might be useful to switch off to another endpoint, although this one is also annoyingly touchy with 403s.
Breakdown
Endpoint: https://clients5.google.com/translate_a/t
Query Parameters

Query Parameter
Default
Notes

client
dict-chrome-ex
Needs to be dict-chrome-ex or else you'll get a 403 error.

sl
auto
Designates the source language of the text to translate.

tl
(none)
Designates the destination language of the text to translate.

q
(none)
Text to translate

Example Response

URL: https://clients5.google.com/translate_a/t?client=dict-chrome-ex&sl=auto&tl=en&q=bonjour

{
  "sentences": [
    {
      "trans": "Hello",
      "orig": "bonjour",
      "backend": 1
    }
  ],
  "dict": [
    {
      "pos": "interjection",
      "terms": [
        "Hello!",
        "Hi!",
        "Good morning!",
        "Good afternoon!",
        "How do you do?",
        "Hallo!",
        "Hullo!",
        "Welcome!"
      ],
      "entry": [
        {
          "word": "Hello!",
          "reverse_translation": [
            "Bonjour!",
            "Salut!",
            "Tiens!",
            "Allô!"
          ],
          "score": 0.7316156
        },
        {
          "word": "Hi!",
          "reverse_translation": [
            "Salut!",
            "Bonjour!",
            "Hé!"
          ],
          "score": 0.084690653
        },
        {
          "word": "Good morning!",
          "reverse_translation": [
            "Bonjour!"
          ],
          "score": 0.065957151
        },
        {
          "word": "Good afternoon!",
          "reverse_translation": [
            "Bonjour!"
          ],
          "score": 0.02749503
        },
        {
          "word": "How do you do?",
          "reverse_translation": [
            "Bonjour!",
            "Salut!",
            "Ça marche?"
          ]
        },
        {
          "word": "Hallo!",
          "reverse_translation": [
            "Bonjour!",
            "Tiens!",
            "Salut!",
            "Allô!"
          ]
        },
        {
          "word": "Hullo!",
          "reverse_translation": [
            "Tiens!",
            "Allô!",
            "Salut!",
            "Bonjour!"
          ]
        },
        {
          "word": "Welcome!",
          "reverse_translation": [
            "Salut!",
            "Bonjour!",
            "Soyez le bienvenu!"
          ]
        }
      ],
      "base_form": "Bonjour!",
      "pos_enum": 9
    }
  ],
  "src": "fr",
  "alternative_translations": [
    {
      "src_phrase": "bonjour",
      "alternative": [
        {
          "word_postproc": "Hello",
          "score": 1000,
          "has_preceding_space": true,
          "attach_to_next_token": false
        }
      ],
      "srcunicodeoffsets": [
        {
          "begin": 0,
          "end": 7
        }
      ],
      "raw_src_segment": "bonjour",
      "start_pos": 0,
      "end_pos": 0
    }
  ],
  "confidence": 0.96875,
  "ld_result": {
    "srclangs": [
      "fr"
    ],
    "srclangs_confidences": [
      0.96875
    ],
    "extended_srclangs": [
      "fr"
    ]
  },
  "query_inflections": [
    {
      "written_form": "bonjour",
      "features": {
        "gender": 1,
        "number": 2
      }
    },
    {
      "written_form": "bonjours",
      "features": {
        "gender": 1,
        "number": 1
      }
    },
    {
      "written_form": "bonjour",
      "features": {
        "number": 2
      }
    },
    {
      "written_form": "bonjours",
      "features": {
        "number": 1
      }
    }
  ]
}

This seems like a great discovery! I'll take a look at this shortly. Thank you so much!

This seems like a great discovery! I'll take a look at this shortly. Thank you so much!

@ssut No problem!

@ssut No problem!

Thanks a lot for this! <3 I've implemented this endpoint myself with requests as googletrans stopped working for me one day.

Thanks a lot for this! <3 I've implemented this endpoint myself with requests as googletrans stopped working for me one day.

@theonefoster Glad to be of use!

@theonefoster Glad to be of use!

Stopped working for me (403)

Stopped working for me (403)

@shatteringlass Mentioned that in the initial post:

it might be useful to switch off to another endpoint, although this one is also annoyingly touchy with 403s.
That's why I want this endpoint to be seamlessly integrated into googletrans, with it switching between endpoints if one is facing 4xx/5xx errors.

@shatteringlass Mentioned that in the initial post:

it might be useful to switch off to another endpoint, although this one is also annoyingly touchy with 403s.
That's why I want this endpoint to be seamlessly integrated into googletrans, with it switching between endpoints if one is facing 4xx/5xx errors.

I found another endpoint within the source code of one of the google translate extensions on VSCode too.
"https://translate.googleapis.com/translate_a/single?client=gtx&dt=t + params"
// where the params are:
{
  "sl": source language,
  "tl": destination language,
  "q": the text to translate
}

The results looks something like this:
[[["こんにちは、今日はお元気ですか？","Hello, how are you today?",null,null,3,null,null,[[]
]
,[[["9588ca5d94759e1e85ee26c1b641b1e3","kgmt_en_ja_2020q3.md"]
]
]
]
]
,null,"en",null,null,null,null,[]
]

for the query: https://translate.googleapis.com/translate_a/single?client=gtx&dt=t&sl=en&tl=ja&q=Hello, how are you today?

And something like this:
[[["Bonjour","Hello",null,null,1]
]
,null,"en",null,null,null,null,[]
]

for the query: https://translate.googleapis.com/translate_a/single?client=gtx&dt=t&sl=en&tl=fr&q=Hello

When using it only to translate things, I would use it like so:
from json import loads
from requests import get
request_result = get("https://translate.googleapis.com/translate_a/single?client=gtx&dt=t&sl=en&tl=fr&q=Hello")
translated_text = loads(request_result.text)[0][0][0]
print(translated_text)

I found another endpoint within the source code of one of the google translate extensions on VSCode too.
"https://translate.googleapis.com/translate_a/single?client=gtx&dt=t + params"
// where the params are:
{
  "sl": source language,
  "tl": destination language,
  "q": the text to translate
}

The results looks something like this:
[[["こんにちは、今日はお元気ですか？","Hello, how are you today?",null,null,3,null,null,[[]
]
,[[["9588ca5d94759e1e85ee26c1b641b1e3","kgmt_en_ja_2020q3.md"]
]
]
]
]
,null,"en",null,null,null,null,[]
]

for the query: https://translate.googleapis.com/translate_a/single?client=gtx&dt=t&sl=en&tl=ja&q=Hello, how are you today?

And something like this:
[[["Bonjour","Hello",null,null,1]
]
,null,"en",null,null,null,null,[]
]

for the query: https://translate.googleapis.com/translate_a/single?client=gtx&dt=t&sl=en&tl=fr&q=Hello

When using it only to translate things, I would use it like so:
from json import loads
from requests import get
request_result = get("https://translate.googleapis.com/translate_a/single?client=gtx&dt=t&sl=en&tl=fr&q=Hello")
translated_text = loads(request_result.text)[0][0][0]
print(translated_text)

@Animenosekai Awesome, another endpoint to add. We should probably create a way to house all of these endpoints through one API, perhaps an interface that all of the endpoints implement?

@Animenosekai Awesome, another endpoint to add. We should probably create a way to house all of these endpoints through one API, perhaps an interface that all of the endpoints implement?

What am I doing wrong?
word = 'لماذا تفعل هذا'
try:
    request_result = requests.post("https://clients5.google.com/translate_a/t?client=dict-chrome-ex&sl=auto&tl=en&q="+ word).json()

    print(request_result)
    print('[In English]: '+ request_result['alternative_translations'][0]['alternative'][0]['word_postproc'])
    print('[Language Dectected]: ' + request_result['src'])
except:
     traceback.print_exc()

Output:
[In English]: Ù „Ù… Ø§Ø ° Ø§ ØªÙ Ø¹Ù„ Ù ‡ Ø ° Ø§
[Language Dectected]: co

why is it outputting that?

What am I doing wrong?
word = 'لماذا تفعل هذا'
try:
    request_result = requests.post("https://clients5.google.com/translate_a/t?client=dict-chrome-ex&sl=auto&tl=en&q="+ word).json()

    print(request_result)
    print('[In English]: '+ request_result['alternative_translations'][0]['alternative'][0]['word_postproc'])
    print('[Language Dectected]: ' + request_result['src'])
except:
     traceback.print_exc()

Output:
[In English]: Ù „Ù… Ø§Ø ° Ø§ ØªÙ Ø¹Ù„ Ù ‡ Ø ° Ø§
[Language Dectected]: co

why is it outputting that?

@NawtJ0sh Why are you making a POST request? It's a GET request you should be making. How does that even return a response?

@NawtJ0sh Why are you making a POST request? It's a GET request you should be making. How does that even return a response?

No idea, it does the exact same thing requests.get()

No idea, it does the exact same thing requests.get()

No idea, it does the exact same thing with requests.get()

If you try to open it in a browser it seems to work fine:
https://clients5.google.com/translate_a/t?client=dict-chrome-ex&sl=auto&tl=en&q=%D9%84%D9%85%D8%A7%D8%B0%D8%A7%20%D8%AA%D9%81%D8%B9%D9%84%20%D9%87%D8%B0%D8%A7
{"sentences":[{"trans":"Why are you doing this","orig":"لماذا تفعل هذا","backend":1},{"src_translit":"limadha tafeal hdha"}],"src":"ar","alternative_translations":[{"src_phrase":"لماذا تفعل هذا","alternative":[{"word_postproc":"Why are you doing this","score":1000,"has_preceding_space":true,"attach_to_next_token":false}],"srcunicodeoffsets":[{"begin":0,"end":14}],"raw_src_segment":"لماذا تفعل هذا","start_pos":0,"end_pos":0}],"confidence":1,"ld_result":{"srclangs":["ar"],"srclangs_confidences":[1],"extended_srclangs":["ar"]}}

Might be a problem with the decoding, try to use the json module yourself (print request.text() first and print the json formatted version json.loads(request.text()) to see the differences)
It might be because your terminal doesn't support arabic or something like that

No idea, it does the exact same thing with requests.get()

If you try to open it in a browser it seems to work fine:
https://clients5.google.com/translate_a/t?client=dict-chrome-ex&sl=auto&tl=en&q=%D9%84%D9%85%D8%A7%D8%B0%D8%A7%20%D8%AA%D9%81%D8%B9%D9%84%20%D9%87%D8%B0%D8%A7
{"sentences":[{"trans":"Why are you doing this","orig":"لماذا تفعل هذا","backend":1},{"src_translit":"limadha tafeal hdha"}],"src":"ar","alternative_translations":[{"src_phrase":"لماذا تفعل هذا","alternative":[{"word_postproc":"Why are you doing this","score":1000,"has_preceding_space":true,"attach_to_next_token":false}],"srcunicodeoffsets":[{"begin":0,"end":14}],"raw_src_segment":"لماذا تفعل هذا","start_pos":0,"end_pos":0}],"confidence":1,"ld_result":{"srclangs":["ar"],"srclangs_confidences":[1],"extended_srclangs":["ar"]}}

Might be a problem with the decoding, try to use the json module yourself (print request.text() first and print the json formatted version json.loads(request.text()) to see the differences)
It might be because your terminal doesn't support arabic or something like that

After some testing with the request headers, I found the solution for the garbled text.
Simply set the User-Agent header to the one that Google Chrome uses.
Example:
import requests
word = 'لماذا تفعل هذا'
url = "https://clients5.google.com/translate_a/t?client=dict-chrome-ex&sl=auto&tl=en&q=" + word
headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.104 Safari/537.36'
}

try:
    request_result = requests.get(url, headers=headers).json()

    print(request_result)
    print('[In English]: ' + request_result['alternative_translations'][0]['alternative'][0]['word_postproc'])
    print('[Language Dectected]: ' + request_result['src'])
except:
     pass

Response:
{'sentences': [{'trans': 'Why are you doing this', 'orig': 'لماذا تفعل هذا', 'backend': 1}, {'src_translit': 'limadha tafeal hdha'}], 'src': 'ar', 'alternative_translations': [{'src_phrase': 'لماذا تفعل هذا', 'alternative': [{'word_postproc': 'Why are you doing this', 'score': 1000, 'has_preceding_space': True, 'attach_to_next_token': False}], 'srcunicodeoffsets': [{'begin': 0, 'end': 14}], 'raw_src_segment': 'لماذا تفعل هذا', 'start_pos': 0, 'end_pos': 0}], 'confidence': 1, 'ld_result': {'srclangs': ['ar'], 'srclangs_confidences': [1], 'extended_srclangs': ['ar']}}
[In English]: Why are you doing this
[Language Dectected]: ar

After some testing with the request headers, I found the solution for the garbled text.
Simply set the User-Agent header to the one that Google Chrome uses.
 It's almost like a form of copy protection.

@d4n3436 How interesting. It's almost like a form of copy protection.

@d4n3436 How interesting. It's almost like a form of copy protection.

Maybe that they are retrieving the User-Agent to process some analytics in the bg and that without it it breaks something.
But, I mean, still weird that it gives a gibberish response and that it accepted the POST request

@d4n3436 How interesting. It's almost like a form of copy protection.

Maybe that they are retrieving the User-Agent to process some analytics in the bg and that without it it breaks something.
But, I mean, still weird that it gives a gibberish response and that it accepted the POST request

Maybe that they are retrieving the User-Agent to process some analytics in the bg and that without it it breaks something.

It's weird...
If you don't use User-Agent, the response will be incorrectly encoded (ASCII).
If you use a banned User-Agent like curl/7.37.1, you'll get a 403 error page.
But if you use a web browser User-Agent, the response will be correctly encoded (UTF-8).
About the POST request, it seems that the API endpoint allows using POST requests using query parameters instead of a body. This also works with PUT, PATCH, DELETE and OPTIONS (basically all common methods).

Maybe that they are retrieving the User-Agent to process some analytics in the bg and that without it it breaks something.

It's weird...
If you don't use User-Agent, the response will be incorrectly encoded (ASCII).
If you use a banned User-Agent like curl/7.37.1, you'll get a 403 error page.
But if you use a web browser User-Agent, the response will be correctly encoded (UTF-8).
About the POST request, it seems that the API endpoint allows using POST requests using query parameters instead of a body. This also works with PUT, PATCH, DELETE and OPTIONS (basically all common methods).

if anyone wants to try and get this Bing translator to work that'd be awesome
import requests

url = 'https://www.bing.com/ttranslatev3'

post_header = {}
post_header['Host'] = 'www.bing.com'
post_header['User-Agent'] = 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0'
post_header['Accept'] = '*/*'
post_header['Accept-Language'] = 'en-US,en;q=0.5'
post_header['Accept-Encoding'] = 'gzip, deflate'
post_header['Referer'] = 'https://www.bing.com/'
post_header['Content-Type'] = 'application/x-www-form-urlencoded'
post_header['Connection'] = 'keep-alive'

parameters_payload = {'IG' : '839D27F8277F4AA3B0EDB83C255D0D70', 'IID' : 'translator.5033.3'}
data_payload = {'text':'Platypus', 'from':'en', 'to':'es'}
resp = requests.post(url, headers=post_header, params=parameters_payload, data=data_payload)

print(resp.json())

if anyone wants to try and get this Bing translator to work that'd be awesome
import requests

url = 'https://www.bing.com/ttranslatev3'

post_header = {}
post_header['Host'] = 'www.bing.com'
post_header['User-Agent'] = 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0'
post_header['Accept'] = '*/*'
post_header['Accept-Language'] = 'en-US,en;q=0.5'
post_header['Accept-Encoding'] = 'gzip, deflate'
post_header['Referer'] = 'https://www.bing.com/'
post_header['Content-Type'] = 'application/x-www-form-urlencoded'
post_header['Connection'] = 'keep-alive'

parameters_payload = {'IG' : '839D27F8277F4AA3B0EDB83C255D0D70', 'IID' : 'translator.5033.3'}
data_payload = {'text':'Platypus', 'from':'en', 'to':'es'}
resp = requests.post(url, headers=post_header, params=parameters_payload, data=data_payload)

print(resp.json())

@NawtJ0sh Microsoft might have changed their API since I get back:
{'statusCode': 400} # if this was a http status code, means bad request --> might be because the params don't work anymore

(which is weird since the HTTP status code is 200)

@NawtJ0sh Microsoft might have changed their API since I get back:
{'statusCode': 400} # if this was a http status code, means bad request --> might be because the params don't work anymore

(which is weird since the HTTP status code is 200)

@NawtJ0sh I've made a simple API wrapper for Bing Translator in C# some months ago. You can use it and port it.
BingTranslatorApi.cs

@NawtJ0sh I've made a simple API wrapper for Bing Translator in C# some months ago. You can use it and port it.
BingTranslatorApi.cs

@NawtJ0sh I made a simple API wrapper for Bing Translator in C# some months ago. You can use it and port it.
BingTranslatorApi.cs

Nice! if you could try it with the Yandex translator, that'd be cool!

@NawtJ0sh I made a simple API wrapper for Bing Translator in C# some months ago. You can use it and port it.
BingTranslatorApi.cs

Nice! if you could try it with the Yandex translator, that'd be cool!

@NawtJ0sh Microsoft might have changed their API since I get back:
{'statusCode': 400} # if this was a http status code, means bad request --> might be because the params don't work anymore

(which is weird since the HTTP status code is 200)

All that was changed was the 'from' to 'fromLang' haha

@NawtJ0sh Microsoft might have changed their API since I get back:
{'statusCode': 400} # if this was a http status code, means bad request --> might be because the params don't work anymore

(which is weird since the HTTP status code is 200)

All that was changed was the 'from' to 'fromLang' haha

Nice! if you could try it with the Yandex translator, that'd be cool!

I actually made one, but unfortunately Yandex has a captcha protection that is triggered after a few uses of their API.

Nice! if you could try it with the Yandex translator, that'd be cool!

I actually made one, but unfortunately Yandex has a captcha protection that is triggered after a few uses of their API.

@d4n3436 oh i see, I meant like a web scraper for it, would that be possible?

@d4n3436 oh i see, I meant like a web scraper for it, would that be possible?

@NawtJ0sh Microsoft might have changed their API since I get back:
{'statusCode': 400} # if this was a http status code, means bad request --> might be because the params don't work anymore

(which is weird since the HTTP status code is 200)

All that was changed was the 'from' to 'fromLang' haha

Lmao, yea I just checked and it worked!
[{'detectedLanguage': {'language': 'en', 'score': 1.0}, 'translations': [{'text': 'Ornitorrinco', 'to': 'es', 'sentLen': {'srcSentLen': [8], 'transSentLen': [12]}}]}]

@NawtJ0sh Microsoft might have changed their API since I get back:
{'statusCode': 400} # if this was a http status code, means bad request --> might be because the params don't work anymore

(which is weird since the HTTP status code is 200)

All that was changed was the 'from' to 'fromLang' haha

Lmao, yea I just checked and it worked!
[{'detectedLanguage': {'language': 'en', 'score': 1.0}, 'translations': [{'text': 'Ornitorrinco', 'to': 'es', 'sentLen': {'srcSentLen': [8], 'transSentLen': [12]}}]}]

if anyone wants to try and get this Bing translator to work that'd be awesome

Well here you go:
from json import loads
from requests import post
HEADERS = {
    "Host": "www.bing.com",
    "User-Agent": "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0",
    "Accept": "*/*",
    "Accept-Language": "en-US,en;q=0.5",
    "Accept-Encoding": "gzip, deflate",
    "Referer": "https://www.bing.com/",
    "Content-Type": "application/x-www-form-urlencoded",
    "Connection": "keep-alive"
}
PARAMS = {'IG' : '839D27F8277F4AA3B0EDB83C255D0D70', 'IID' : 'translator.5033.3'}

def translate(text, to, fromLang="auto-detect"):
    """
    Translates the given text to the given language using Microsoft's Bing Translation API (ttranslatev3)
    """
    request = post("https://www.bing.com/ttranslatev3", headers=HEADERS, params=PARAMS, data={'text': str(text), 'fromLang': str(fromLang), 'to': str(to)})
    if request.status_code < 400:
        try:
            return loads(request.text)[0]["translations"][0]["text"]
        except:
            return None
    else:
        return None

def language(text):
    """
    Gives you back the ISO 639-1 Alpha-2 language code the text has been written in using Microsoft's Bing Translation API (ttranslatev3)

    > The output is a tuple with the language code and the score (confidence)
    """
    request = post("https://www.bing.com/ttranslatev3", headers=HEADERS, params=PARAMS, data={'text': str(text), 'fromLang': "auto-detect", 'to': "en"})
    if request.status_code < 400:
        try:
            detectedLanguage = loads(request.text)[0]["detectedLanguage"]
            return detectedLanguage["language"], detectedLanguage["score"]
        except:
            return None
    else:
        return None

Just call
translate("<your text>", "<the ISO 639-1 Alpha-2 language codes (I guess) of the output language>")

Or if you want to know the language of a given text:
language("<your text>")

And it gives you back the result or None if an error occurred.

Example:

>>> translate("Hello", "ja")
'こんにちは'
>>> translate("Hello", "fr")
'Bonjour'
>>> translate("Hello", "fr", "en")
'Bonjour'
>>> language("Hello")
('en', 1.0)
>>> language("Hola")
('es', 1.0)

if anyone wants to try and get this Bing translator to work that'd be awesome

Well here you go:
from json import loads
from requests import post
HEADERS = {
    "Host": "www.bing.com",
    "User-Agent": "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0",
    "Accept": "*/*",
    "Accept-Language": "en-US,en;q=0.5",
    "Accept-Encoding": "gzip, deflate",
    "Referer": "https://www.bing.com/",
    "Content-Type": "application/x-www-form-urlencoded",
    "Connection": "keep-alive"
}
PARAMS = {'IG' : '839D27F8277F4AA3B0EDB83C255D0D70', 'IID' : 'translator.5033.3'}

def translate(text, to, fromLang="auto-detect"):
    """
    Translates the given text to the given language using Microsoft's Bing Translation API (ttranslatev3)
    """
    request = post("https://www.bing.com/ttranslatev3", headers=HEADERS, params=PARAMS, data={'text': str(text), 'fromLang': str(fromLang), 'to': str(to)})
    if request.status_code < 400:
        try:
            return loads(request.text)[0]["translations"][0]["text"]
        except:
            return None
    else:
        return None

def language(text):
    """
    Gives you back the ISO 639-1 Alpha-2 language code the text has been written in using Microsoft's Bing Translation API (ttranslatev3)

    > The output is a tuple with the language code and the score (confidence)
    """
    request = post("https://www.bing.com/ttranslatev3", headers=HEADERS, params=PARAMS, data={'text': str(text), 'fromLang': "auto-detect", 'to': "en"})
    if request.status_code < 400:
        try:
            detectedLanguage = loads(request.text)[0]["detectedLanguage"]
            return detectedLanguage["language"], detectedLanguage["score"]
        except:
            return None
    else:
        return None

Just call
translate("<your text>", "<the ISO 639-1 Alpha-2 language codes (I guess) of the output language>")

Or if you want to know the language of a given text:
language("<your text>")

And it gives you back the result or None if an error occurred.

Example:

>>> translate("Hello", "ja")
'こんにちは'
>>> translate("Hello", "fr")
'Bonjour'
>>> translate("Hello", "fr", "en")
'Bonjour'
>>> language("Hello")
('en', 1.0)
>>> language("Hola")
('es', 1.0)

Here is the yandex translate, I worked on last night its just like the bing.com one i posted.
import requests

url = 'https://translate.yandex.net/api/v1/tr.json/translate?id=1308a84a.6016deed.0c4881a2.74722d74657874-3-0&srv=tr-text&lang=en&reason=auto&format=text'

post_header = {}
post_header['Accept'] = '*/*'
post_header['Accept-Encoding'] = 'gzip, deflate'
post_header['Accept-Language'] = 'en-US,en;q=0.9'
post_header['Cache-Control'] = 'no-cache'
post_header['Connection'] = 'keep-alive'
post_header['Content-Type'] = 'application/x-www-form-urlencoded'
post_header['Host'] = 'translate.yandex.com'
post_header['Referer'] = 'https://translate.yandex.com/'
post_header['User-Agent'] = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.104 Safari/537.36'

data_payload = {'text': search_str, 'options': '4'}

resp = requests.get(url, headers=post_header, data=data_payload).json()

print(str(resp) + '\n')

@NawtJ0sh
I just made a repository which groups all of the APIs if you want (called translate)

Nice you fixed it!! Thank you.

The endpoint now returns a simpler response (translation and detected language).
Url: https://clients5.google.com/translate_a/t?client=dict-chrome-ex&sl=auto&tl=es&q=Hello World
Response:
[[[[["Hola Mundo"]],null,"en"]]]

@d4n3436 It seems our little operation has been busted by Google. This heavily reminds me of their other obfuscated JSON responses, where arrays are used instead of objects, which makes it hell for us stragglers to trivially parse. Try making more translations in a variety of languages and then share the results with us, as the lengths of the inner arrays may vary.

@SuperSonicHub1 I ran a test program that translates a long text into all the supported languages and the response structure didn't change.

Yeah that Google Dictionary Chrome extension doesn't seem to be working now lol because of this.

Yeah that Google Dictionary Chrome extension doesn't seem to be working now lol because of this.

Do you see any direct issues caused by this on translatepy or does it handle it correctly? (I will test it by myself as soon as possible)

The endpoint now returns a simpler response (translation and detected language).
Url: https://clients5.google.com/translate_a/t?client=dict-chrome-ex&sl=auto&tl=es&q=Hello World
Response:
[[[[["Hola Mundo"]],null,"en"]]]
Formatted response:
[
    [
        [
            [
                [
                    "Hola Mundo"
                ]
            ],
            null,
            "en"
        ]
    ]
]

I can confirm.

@d4n3436 It seems our little operation has been busted by Google. This heavily reminds me of their other obfuscated JSON responses, where arrays are used instead of objects, which makes it hell for us stragglers to trivially parse. Try making more translations in a variety of languages and then share the results with us, as the lengths of the inner arrays may vary.

An example for Chinese (some random item from a number of files). The input was 语言与词汇的关系. Nesting and structure appears to be the same as for Spanish.
[
  [
    [
      [
        [
          "The relationship between language and vocabulary"
        ]
      ],
      null,
      "zh-CN"
    ]
  ]
]

Well but I mean the most important reminds: we still have the translation and the detected language so it shouldn't be too hard because the scheme is not as obfuscated as the API used nowadays on the Google Translate website (batchexecute)

Well but I mean the most important reminds: we still have the translation and the detected language so it shouldn't be too hard because the scheme is not as obfuscated as the API used nowadays on the Google Translate website (batchexecute)

Yes, and if you are translating in bulk you can just name the output files in a manner that allows you to connect your input with the returned json files, I use simple hash keys. Does any version of the googletrans package allow to pull the extra info these days? I haven't used it in a while. SuperSonicHub1's API URL with bash command line utilities were perfect for my needs.
Happy translating, everyone!

Yeah that Google Dictionary Chrome extension doesn't seem to be working now lol because of this.

Do you see any direct issues caused by this on translatepy or does it handle it correctly? (I will test it by myself as soon as possible)

Have you tested it? To see if there is any errors??

Have you tested it? To see if there is any errors??

There doesn't seem to have any errors for now as it is using the other endpoint first:

But I'll try to investigate more into this

Do you see any direct issues caused by this on translatepy or does it handle it correctly? (I will test it by myself as soon as possible)

Have you tested it? To see if there is any errors??

I've uploaded a new version of translatepy with the following fix:
return data[0][0][2], "".join(sentence for sentence in data[0][0][0][0])

which makes it work with this new response format.
Update translatepy with pip install --upgrade translatepy (it should install translatepy v1.5.1)

Seems like a few hours ago @SuperSonicHub1's Google Translate endpoint has started to return the more detailed response structure.

Seems like a few hours ago @SuperSonicHub1's Google Translate endpoint has started to return the more detailed response structure.

... I'll look into it

Wow, seems like its changed back to how it was before...

Wow, seems like its changed back to how it was before...
Could do this....
try:
    return data[0][0][2], "".join(sentence for sentence in data[0][0][0][0])
except:
    return data['ld_result']["srclangs"][0], "".join(sentence["trans"] for sentence in data["sentences"])

Lmao that's exactly what I did on Animenosekai/translate d5f0357 (flipped their position though)

The endpoint now returns a simpler response (translation and detected language).
Url: https://clients5.google.com/translate_a/t?client=dict-chrome-ex&sl=auto&tl=es&q=Hello World
Response:
[[[[["Hola Mundo"]],null,"en"]]]
Formatted response:
[
    [
        [
            [
                [
                    "Hola Mundo"
                ]
            ],
            null,
            "en"
        ]
    ]
]

This could fix your issue. It's a bit janky, but it works.
`import requests
from urllib.parse import urlencode
##TIMEOUT = 30
TIMEOUT = 3
MAGICHEADERS = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.104 Safari/537.36'
}
class ResponseError(Exception):
pass
def get_translation_url(sentance, tolanguage, fromlanguage='auto'):
"""Return the url you should visit to get sentance translated to language tolanguage."""
query = {'client': 'gtx',
'dt'    : 't',
'sl'    : fromlanguage,
'tl'    : tolanguage,
'q'     : sentance}
url = 'https://translate.googleapis.com/translate_a/single?'+urlencode(query)
return url
def translate(sentance, tolang, fromlang='auto'):
"""Use the power of sneeky tricks to do translation."""
# Get url from function, which uses urllib to generate proper query
url = get_translation_url(sentance, tolang, fromlang)
try:
# Make a get request to translate url with magic headers
# that make it work right cause google is smart and looks at that.
# Also, make request result be json so we can look at it easily
request_result = requests.get(url, headers=MAGICHEADERS).json()
except Exception as ex:
# If it broke somehow, try again
# NOTE: could cause indefinite recursion
return translate(sentance, tolang, fromlang)
# After we get the result, get the right field of the response and return that.
# If result field not in request result
def get_parts(lst):
usefull = []
for i in lst:
if isinstance(i, list):
usefull += get_parts(i)
elif i:
usefull.append(i)
return usefull
return get_parts(request_result)[0]`
But I have noticed that quite quickly google recognizes similarities and stops it from continued use, but maybe that's just me testing the same sort of general sentence repeating a couple of times.

import requests
"""
l = language
w = word
"""
def trans1(l, w):
r = requests.get(f"https://translate.googleapis.com/translate_a/single?client=gtx&dt=t&sl=en&tl={l}&q={w}", headers=h)
# print(r)
# print(r.text.encode('utf-8'))
if r.status_code == 200:
translated_text = r.json()[0][0][0]
return(translated_text)
def trans2(l, w):
r = requests.get(f"https://clients5.google.com/translate_a/t?client=dict-chrome-ex&sl=auto&tl={l}&q={w}", headers=h)
# print(r)
# print(r.text.encode('utf-8'))
if r.status_code == 200:
translated_text = r.json()["sentences"][0]["trans"]
return(translated_text)

Error from server : Too many request :'(

Closing this issue as our research was never implemented in this repo. I think it's best to assume that py-googletrans is pretty much done. Still need a translation library? Check out these:

https://github.com/nidhaloff/deep-translator
https://github.com/Animenosekai/translate

Please stop using this issue as a support thread; tired of getting everyone's emails. Maintainers, I'd recommend you lock this thread.

Why you confusing people with broken package? Update the readme and unpublish from the pip. Thats it

It seems that now the two "free endpoints" of Google Translate API don't respond in a nice way anymore. For single words translations, they used to respond with a detailed list of possible translations. Now they only give you back a single one (the most common one) and I can't find a way to change this.
Have a look for "body" translated from EN to IT ("corpo", "organismo", "organo", "carrozzeria", ...)
Free endpoint 1: https://clients5.google.com/translate_a/t?client=dict-chrome-ex&sl=en&tl=it&q=body
Free endpoint 2: https://translate.googleapis.com/translate_a/single?client=gtx&dt=t&sl=en&tl=it&q=body
Any ideas?

I don't believe it would be possible to get detailed lists without using a different endpoint, because in the applications these endpoints are in, they don't need all the possible translations.

I've found this alternative endpoint for google:
Free endpoint: https://translate.google.com/m?hl=en&sl=en&tl=pt&ie=UTF-8&prev=_m&q=Text
The caveat? the response is an HTML page. It is easily parseable though

@lokinmodar The only caveat is that it's the mobile version of Google Translate which tends to yield worse translations

@lokinmodar The only caveat is that it's the mobile version of Google Translate which tends to yield worse translations

@lokinmodar The only caveat is that it's the mobile version of Google Translate which tends to yield worse translations (based on previous observations, this might have changed and this page is slightly different than the normal Google Translate mobile page)

i noticed this is much more faithful to the translations I used to get with the https://clients5.google.com/translate_a/t?client=dict-chrome-ex&sl=en&tl=it&q=body endpoint before they screwed it up

I made a comparison of all (known) Google Translate API endpoints based on my findings while making my translation library, this may be useful if you don't know which API endpoint to use.

https://clients5.google.com/translate_a no longer seems to return faithful translations; only returns the translated text.
https://translate.googleapis.com/translate_a/single still returns faithful translations and some info about the translation (source language, translation confidence and transliteration if available).
https://translate.google.com/_/TranslateWebserverUi/data/batchexecute only returns a faithful translation with the X-Goog-BatchExecute-Bgr, the algorithm that creates this header doesn't seem to be deciphered yet. This is the endpoint that returns the most info (source language, translation confidence, transliteration, alternative translations, definitions, examples, etc.).
https://translate.google.com/m returns a faithful translation, but no extra info. Returns HTML instead of JSON so parsing is required.

In conclusion, the second endpoint is currently the best free Google Translate API endpoint, unless someone manages to decipher the X-Goog-BatchExecute-Bgr header.

''' https://clients5.google.com/translate_a/single?dj=1&dt=t&dt=sp&dt=ld&dt=bd&client=dict-chrome-ex&sl=auto&tl=en&q=kanker ''' works

Does anyone understand what is "score". Looks like it show how popular translation. I can't understand how to convert it to dashes like google translate does.

https://translate.google.com/_/TranslateWebserverUi/data/batchexecu seems no longer to work, 404 now.

https://translate.google.com/_/TranslateWebserverUi/data/batchexecute still OK. https://translate.google.cn/_/TranslateWebserverUi/data/batchexecute no longer works because translate.google.cn is dead (redirected to translate.google.com.hk in fact).

Hello,everyone
someone who can find translate from image.

I know this is an older thread but just wanted to warn people that https://translate.googleapis.com/translate_a/single has issues with punctuation and will just halt translations (like if it encounters an '!').
https://clients5.google.com/translate_a/t works great though! Thank you @SuperSonicHub1 !

I know this is an older thread but just wanted to warn people that https://translate.googleapis.com/translate_a/single has issues with punctuation and will just halt translations (like if it encounters an '!').
https://clients5.google.com/translate_a/t works great though! Thank you @SuperSonicHub1 !

I have found that  https://translate.googleapis.com/translate_a/single works perfectly, even when it encounters punctuation such as "!". The only drawback is that it splits the translation into multiple parts if the query is too long, which requires us to join them back together. Additionally, it provides better translations compared to  https://clients5.google.com/translate_a/t.


Users report not getting a CSV download option once completing the checklist

Needs analysis and QA

waiting for detailed user feedback of the issue

user feedback: When Results page presented, Download CSV link not obviously visible. Usability issue
Solution: Add Details of how to download Results as CSV at top of Results page when displayed

Waiting for user feedback

user verified success


ResizeObserver

Supported on Chrome 54+, in development on Firefox
https://www.chromestatus.com/feature/5705346022637568
https://developers.google.com/web/updates/2016/10/resizeobserver
Proposal: https://wicg.github.io/ResizeObserver/

Duplicate of #3006


receive a post method of json request.

Looks like a good start - It would be nice to try and implement a more transparent solution but that can be deferred.

Covered by #1173


Dataview auto-generated code blocks clicks

Dataviews seem to auto-generate this code:
"click td": function(e, t) {
	e.preventDefault();
	/**/
	return false;
},

This is unfortunate as I have embedded HTML links in the dataview that the user should be able to click on. The above blocks these clicks.
Is there a good workaround?  I don't see any way to handle it in the Kitchen GUI.

I figured it out, so ignore this one.
I can create an overriding "click td a" in an extra JS file that handles the click on the link. It works perfectly well.

@xauxatz excuse me for too late answer. Glad you solve it! :+1:


The NCoC Readme contains multiple instances codifying expected conduct. These should be removed.

Once again, we are not going to tolerate our community being overridden by the mob. If this starts to happen, we will nuke, delete, lock, close, ban, and do whatever we have to do to put the fire out. These discussions drag on and on and on, and they don't make communities better.

Either clarify what the expected behaviour is, or remove this

Off topic chatter should be discouraged, but not censored. NCoC is targeting things like open source projects and public online communities based around specific topics.

This directly contradicts the above. If you are opposed to censorship then even conversation is "overtaken by a mob" must be allowed to continue unabated, as it is the preference of the majority.
We're all adults here after all!

I was attracted to this repo because of its purity, but, like @wilcornish , I'm realizing that while the three "rules" or agreements, or whatever they are are consistent with not having a CoC, some of the answers in the FAQ are, in fact, spelling out codes of conduct, if not explicitly calling them such.
It seems to me that this repo is not no code of conduct, but a code of conduct of "staying on topic," which is fraught with the same pitfalls, inconsistencies, etc. as all the other codes of conduct out there.  Shame.  You almost had something here.
One could have predicted it: you're tired of people whining about others' behavior, and you're further tired of people putting up codes of conduct to try to protect these whiners or make them feel better.  So to be rid of all of this, you think, "ah, no code of conduct."  But that's a code of conduct!  I sympathize, and if feels good, but ultimately I also don't like it.  I think I may create a true NCoC, maybe call it Truly No Code of Conduct.  Or, you could champion that here, and stop false advertising.

Having a "true" No Code of Conduct doesn't make sense in my opinion.  If a project really wants to have no CoC – it wouldn't include such.  As far as I understand the No Code of Conduct is named as such as most other CoCs out there specify too many details no pertaining to project development.  So I like NCoC clarifies the expected CoC is strictly related to a project's development, which actually may vary across projects.

The way I see it, not having a code of conduct may be due to unawareness (word?), ignorance, laziness, neglect, etc., and if not, it's likely to be perceived that way.  Having a true no code of conduct is, instead, a positive statement that there is no code of conduct.  It also encourages others to proclaim no code of conduct--that the internet should be a free speech zone.
Again, if you want everyone to only discuss the project, then call it a No Off-topic Code of Conduct, or some such.  The intent of the No Code of Conduct here is confusing, as @wilcornish has pointed out.  At once, they say they will not censor, but also say they will

nuke, delete, lock, close, ban, and do whatever we have to do to put the fire out

So the line is very blurry.  There won't be any censorship, unless, one presumes, the off-topic conversation goes "too long," and then it will be.  I call this a lenient No Off-topic Code of Conduct.

The point being made here isn't that we don't expect certain conduct, because people always have certain expectations, but that we do not believe it necessary to have a formal Code of Conduct for open source projects.


how to add custom operator with logic ?

I want to add custom operator with logic .
For example ,

const expr = `` ;
cosnt ast = parse(expr) ;
evaluate(ast,{});

In this example, I want it to return 1024, which is 2 to the 10th power, like the "^" operator in Excel.
I tried creating an original plugin like below .

const myPlugin = {
  name:"my plugin",
  init(){
    jsep.addBinaryOp("^",10);
  }
}
jsep.plugins.register(myPlugin)

However, I don't know where to write the essential logic of "finding exponentiation".
Please tell me how to add a custom operator with logic

Was self resolved.
I was always looking at require("jse-eval").jsep instead of require("jse-eval").
I read the README and realized that I should use addEvaluator.

Sorry for the confusion @TBSten , but glad you figured it out! I'll try to update the readme with some better examples for extending functionality and your use case would be a great example 🙂

Thank you for your response!


[Update Snap] Leap Cosmos Wallet

Checklist
All items in the list below needs to be satisfied.

[ ] Is the summary of the change documented in this ticket?
[ ] Has a MetaMask Snaps team member reviewed whether the changes need to be vetted?
[ ] If there are changes that need to be vetted, attach a description and the relevant fixes/remediations to this issue.
[ ] The corresponding pull request in this repo has been merged.

Summary of changes:

Updated logo https://github.com/leapwallet/cosmos-metamask-snap/pull/43/files#diff-115c03bf4cb102e5c20991b2e71506870f85a872c4adc91760025e54a1dee854
Updated README https://github.com/leapwallet/cosmos-metamask-snap/pull/43/files#diff-342965aca989dae2e06ca2c8c8504869d84d87163ab75596e6ec8e8c5ffbf950
Updated LICENSE https://github.com/leapwallet/cosmos-metamask-snap/pull/43/files#diff-12aa4a290ae875c12f44390cedf17935c6110ab6c2008f20f7a739e499827625
Improve readability of tx signing page https://github.com/leapwallet/cosmos-metamask-snap/pull/43/files#diff-237f4d452a9ca98f8aa9e112288e1a916ae3a1a030da73d15ff5db0f5450d3fb


Add support for command WAIT

Add support for the WAIT command in DiceDB similar to the WAIT command in Redis. Please refer to the following commit in Redis to understand the implementation specifics - source.
Write unit and integration tests for the command referring to the tests written in the Redis codebase 7.2.5. For integration tests, you can refer to the tests folder. Note: they have used TCL for the test suite, and we need to port that to our way of writing integration tests using the relevant helper methods. Please refer to our tests directory.
For the command, benchmark the code and measure the time taken and memory allocs using benchmem and try to keep them to the bare minimum.

@arpitbbhayani  could you please assign this to me?

@raghavbabbar Go for it.

@raghavbabbar could you please add updates on your progress on the issue?

https://discord.com/channels/1034342738960855120/1264145943884992595/1273975330780221522 As we don't have replica at the moment. what functionality are we expecting out from this issue. could you guys please help me @JyotinderSingh

https://discord.com/channels/1034342738960855120/1264145943884992595/1273975330780221522 As we don't have replica at the moment. what functionality are we expecting out from this issue. could you please help me @JyotinderSingh

Makes sense, we can add this issue to the icebox for now.


Be silent when there are no tests to run

In a multi-module project, when I run test in the root which aggregates everything else, I want to be able to hide all the noise about modules without tests.
This is the kind of noise I want to opt-out of.
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] No tests to run for blah-server-schema/test:test
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] No tests to run for blah-server-logic/test:test
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] No tests to run for blah-api/test:test
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] No tests to run for blah-server-impl/test:test
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] No tests to run for blah-server/test:test
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] No tests to run for blah/test:test

I think a normal scoped setting would be enough. Does that sound good? Should I start writing a patch?

HI,
Using version 0.13.11 still can't achieve to silent useless information. How can I do it ?


Suggested improvement for make.sh : add go get  statements

To improve make.sh to handle lack of libraries, consider adding
go get github.com/blang/semver
go get github.com/boltdb/bolt
go get github.com/burntsushi/toml
go get github.com/cheggaaa/pb

in your make.sh

Hey, just a quick advice, go get ./... can do the job instead of 4 lines.
So it, yes it's the same as writing the 4 go get lines, it's just quicker. :)


fixModify the default conf values of mem_limit and cache_last_version_interval_second

Proposed changes
mem_limit from 80% to 90%
cache_last_version_interval_second from 900 to 30
Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

run buildall


Add questionnaire id and remove case id

This PR adds the questionnaire id to the claims to launch eq with and removes the case id.
It is to be used to test the following PR: https://github.com/ONSdigital/eq-survey-runner/pull/2084
https://trello.com/c/0fkw3X5F/2984-jwt-claims-questionnaireid-and-caseid

We should migrate this PR to https://github.com/ONSdigital/eq-questionnaire-launcher

We should migrate this PR to https://github.com/ONSdigital/eq-questionnaire-launcher

It has been already: https://github.com/ONSdigital/eq-questionnaire-launcher/pull/2


Add FIWARE stylesheet to ReadTheDocs

REVIEWERS
@jesuspg
DESCRIPTION

Add FIWARE stylesheet to ReadTheDocs.

TESTING
Remotely in ReadTheDocs the corresponding issue/readthedocs branch.

LGTM


Implement RegExp.prototype.flags

Description
When I try to get the flags of a regExp object, chakra returns undefined. Then I found that chakra not fully implements regular expression prototype at here. Maybe RegExp.prototype.flags should be added too.
Testcase
function f0(){
    return /abc/igm.flags;
}
print(f0());

Version
chakra-_1_11_22(release)

Output
undefined

Expected Output
gim

Contributor:@QuXing9

Works with the relevant flag - enabling this feature is covered by #6390


Removed chrome extension page

chrome web store page is not found, so i cannot import this extenstion :(
https://chrome.google.com/webstore/detail/animal-crossing-music/fcedlaimpcfgpnfdgjbmmfibkklpioop

You have to download/install it directly from github, google took it down, cf discord / twitter

You have to download/install it directly from github, google took it down, cf discord / twitter

Here's a guide: https://github.com/animal-crossing-music-extension/ac-music-extension#installation


should.be.null and should.be.undefined?

Right now I am using should.equal null and should.equal undefined.
Would it be better if you could use these assertions directly?
UPDATE: I just realized that you cannot have methods to null and undefined :)

haha yeah that's a big no unfortunately, for null or undefined checks I usually use node's assert since you can just do assert(!err) etc

I struggled to write the should statement for undefined tests. The following doesn't work.
    target.should.be.undefined();

I found the following solutions.
    (target === undefined).should.be.true()

if can also write it as a type check
   (typeof target).should.be.equal('undefined');

Not sure if the above is the right way, but it does work.

You can use this:
should(target).not.be.ok;


sourceComments option gets ignored if called by gulp.watch

Example:
var gulp = require('gulp');
var sass = require('gulp-sass');

gulp.task('sass', function() {
	gulp.src('**/*.{scss,sass}')
		.pipe(sass({
			sourceComments: true,
			outputStyle: 'expanded',
			errLogToConsole: true
		}))
		.pipe(gulp.dest('screen.css'));
});

gulp.task('watch', ['sass'], function() {
	var sassWatcher = gulp.watch('**/*.{scss,sass}', ['sass']);
});

When I call gulp sass it generates source comments, as expected. But when I run gulp watch no source comments get generated.

Please try with the latest version:
npm install gulp-sass@next --save-dev

Same result.

I cannot replicate your error using the latest version of gulp-sass, in fact. I very clearly see the comments, including on watch, but it brings up an interesting question we should resolve.
Because we're passing a string to node-sass and not a path, the sourceComment says it's coming from stdin for anything coming from the main file. Should we replace this with the path to that file before passing passing back along @dlmanning @Keats? process.cwd() replaced below for the absolute path to the Gulp file.
/* line 1, {{process.cwd()}}/sass/_foo.scss */
.banana {
  content: 'waldo'; }

/* line 3, stdin */
.foo {
  content: 'bar'; }

/* line 7, stdin */
.bar {
  content: 'baz'; }

/* line 11, stdin */
.baz {
  content: 'baz'; }

/* line 15, stdin */
.qux {
  content: 'qux'; }

I'm also happy to defer this to 2.1 as most people seem to be interested in Source Maps over Comments and I don't want to block the 2.0 release on this

I'm ok with moving this to 2.1. I'll take a look at this again, I could bet in my case there where no source comments at all.

I didn't even know that was a thing (I use compressed output with no sourcemaps/anything).
We can definitely work on that for 2.1

For those of you, like me, who are to impatient to wait for a fix you can hack around this by tapping into the stream with [gulp-tap] (https://www.npmjs.com/package/gulp-tap) and just doing plain ol' string replacement. It will likely slow things down as we have to toString the buffer then convert it back but it gets you out of a bind.
gulp.src(config.files.scss)
.pipe(sass({
    sourceComments:true
}))
.pipe(tap(function(file, t) {
    file.contents = new Buffer(file.contents.toString().replace(/stdin \*\//g, file.path + ' */'));
}))
.pipe(concat('app.css'))
}))

This appears to be fix in v2.3.1.

In v2.3.2 it still doesn't work.


Compiling widgetset with addon located in the same source base as project

Hello!
Are there any possibility to point plugin in any way to scan project source path in addition to dependencies in a build file? We just want to avoid a separation of 2 classes to a deicated addon/library.

What do you mean by "Scan the classpath"?
If you put your addon client side classes of your addon in the client package of the project and add a widgetset to your project then the addons will be available in your project without  packaging them in as addons. Is this what you meant?

Exactly.

@maksym72
The newly released 0.11.4 should improve this. If you split out the addon/library into its own module and add a dependency to that module then the plugin will now find the widgetset.
For an example you could have a look at the new wiki article about building addons at https://github.com/johndevs/gradle-vaadin-plugin/wiki/Building-addons and if you need an example you can run the vaadinCreateAddonProject task that will create a project that has an addon and a demo project that depends on that addon.


authentication callback error for application behind istio-ingress-gateway

An application behind istio-ingress gateway receives an error in the authentication callback
[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Extensions and Telemetry
[ x] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
[ ] Upgrade
The application asks for user authentication and receives a callback with jwt token
Bug description
Framework

private cesrtification authority (CA)
all certificates involved are signed by this CA
CA certificate is installed in every node operating system, in every machine involved and in every browser
CA certificate is added to istio operator (without this authentication don't work see bug 29366
the keycloak authentication server is reachable at "https://k6k.m.net/auth/realms/m-project"
the istio ingress url is "https://www.m.net"
the application name is fpresenter

Istio configuration relevant files
RequestAuthentication
sysop@m-serv:~/software/m-project$ cat m-ra.yaml
apiVersion: security.istio.io/v1beta1
kind: RequestAuthentication
metadata:
  name:  m-ra
  namespace: istio-system
spec:
  selector:
    matchLabels:
      istio: ingressgateway
  jwtRules:
  - issuer: "https://k6k.m.net/auth/realms/m-project"
    jwksUri: "https://k6k.m.net/auth/realms/m-project/protocol/openid-connect/certs"
    forwardOriginalToken: true
    outputPayloadToHeader: x-jwt-payload

sysop@m-serv:~/software/m-project$

The first lines of www.m-net virtual service are:
sysop@m-serv:~/software/m-project$ cat m-gateway.yaml m-vservice.yaml
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: mproject-gateway
spec:
  selector:
    istio: ingressgateway # use istio default ingress gateway
  servers:
  - port:
      number: 443
      name: https
      protocol: HTTPS
    tls:
      mode: SIMPLE
      credentialName: www-ingress-tls # must be the same as secret
    hosts:
    - www.m.net

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: fservices
spec:
  hosts:
  - "www.m.net"
  gateways:
  - mproject-gateway
  http:
  - match:
    - uri:
        prefix: "/pres"
    route:
    - destination:
        port:
          number: 3000
        host: fpresenter
  - match:
    - uri:
        prefix: "/presstatic"
    route:
    - destination:
        port:
          number: 3000
        host: fpresenter
  - match:
    - uri:
        exact: "/"
    route:
    - destination:
        port:
          number: 3000
        host: fpresenter
  - match:
    - uri:
        exact: "/logout"
    route:
    - destination:
        port:
          number: 3000
        host: fpresenter
  - match:
    - uri:
        prefix: "/user"
    rewrite:
      uri: "/headers"
    route:
    - destination:
        port:
          number: 8000
        host: fuser

[other services definitions not implied in this report]

Test execution

access unprotected page "https://www.m.net": the page shows correctly
access protected page "https://www.m.net/pres/secured": the application redirect to the keycloak server
the user authenticate with keycloak and is redirected to url "https://www.m.net/pres/secured?auth_callback=1&state=1579e275-0ea8-4db2-809a-4d0d7c809758&session_state=1234e0e1-9f4d-4bca-a68d-e8a177d24488&code=1381c0ff-7b90-4ee1-8304-f16c853f2457.1234e0e1-9f4d-4bca-a68d-e8a177d24488.e624b94f-f568-4249-9cb0-73f8c25b3955"
instead of showing the redirected page, you receive an access denied error

In the application log you see:
GET /pres/secured 302 296.581 ms - 548
Could not obtain grant code: Error: Client network socket disconnected before secure TLS connection was established
GET /pres/secured?auth_callback=1&state=1579e275-0ea8-4db2-809a-4d0d7c809758&session_state=1234e0e1-9f4d-4bca-a68d-e8a177d24488&code=1381c0ff-7b90-4ee1-8304-f16c853f2457.1234e0e1-9f4d-4bca-a68d-e8a177d24488.e624b94f-f568-4249-9cb0-73f8c25b3955 403 59.105 ms - -

To test in a simpler framework I have

an apache2 server (url https://m-serv.m.net ) acting as reverse proxy for port 3000
a docker container running the same image deployed in kubernetes

In these conditions the test gives the results

access unprotected page "https://m-serv.m.net": the page shows correctly
access protected page "https://m-serv.m.net/pres/secured": the application redirect to the keycloak server
the user authenticate with keycloak and is redirected to url "https://m-serv.m.net/pres/secured" and the protected page is shown

In this case the callback is visible only in the logs:
GET /pres/secured?auth_callback=1&state=307c0161-192a-418e-82fd-c36b22513887&session_state=8120c770-6763-4cb5-a84e-32fd193422c2&code=8c790088-87e6-46d1-bcb1-dd76764dd969.8120c770-6763-4cb5-a84e-32fd193422c2.e624b94f-f568-4249-9cb0-73f8c25b3955 302 104.896 ms - 70

Logs for istio-proxy/application (error) and apache2/application( works correctly)
istio_log.txt
Version (include the output of istioctl version --remote and kubectl version --short and helm version --short if you used Helm)
sysop@m-serv:~$ istioctl version --remote
client version: 1.9.3
control plane version: 1.9.3
data plane version: 1.9.3 (6 proxies)
sysop@m-serv:~$ kubectl version --short
Client Version: v1.20.4
Server Version: v1.20.6+k3s1
sysop@m-serv:~$

How was Istio installed?
Istio 1.9.3 is installed using istio operator and setting a private CA certificate (as in bug 29366 solution)
Environment where the bug was observed (cloud vendor, OS, etc)
3 KVM virtual machines with Ubuntu 20.04
K3S master and two workers on the 3 virtual machines
Additionally, please consider running istioctl bug-report and attach the generated cluster-state tarball to this issue.
Refer cluster state archive for more details.
UPDATE 2021/04/26
Loking at the itstio-ingressgateway logs the operations sequence was:
1) user browser ask for "https://www.m.net/pres/secured"
2) istio-ingressgateway forward the request to the fpresenter application and in the istio-ingressgateway logs I see:
[2021-04-26T16:44:48.395Z] "GET /pres/secured HTTP/2" 302 - via_upstream - "-" 0 548 44 44 "<IP_ADDRESS>" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:88.0) Gecko/20100101 Firefox/88.0" "387d5cd4-5769-9188-91a7-56e33fb77c25" "www.m.net" "<IP_ADDRESS>:3000" outbound|3000||fpresenter.m-project.svc.cluster.local <IP_ADDRESS>:51784 <IP_ADDRESS>:8443 <IP_ADDRESS>:28398 www.m.net -

3) the fpresenter application redirects to the keycloak sever at "https://k6k.m.net/auth/realms/m-project".
4) the keycloak login windows shows up and the user can authenticate
5) the keycloak server sends back the to the "https://www.m.net/pres/secured" url adding the jwt token but this cause an error and in the user browser arriva an "Access denied" error. In the istio-ingressgateway logs I see:
[2021-04-26T16:44:52.422Z] "GET /pres/secured?auth_callback=1&state=4bc93ed1-dcb8-4282-88ca-c92c9cdc8011&session_state=774d1a0e-cd3f-4ffe-bb02-615ea3c8f987&code=0c49634e-8ea5-4ec7-b771-c378eabfc8d2.774d1a0e-cd3f-4ffe-bb02-615ea3c8f987.e624b94f-f568-4249-9cb0-73f8c25b3955 HTTP/2" 403 - via_upstream - "-" 0 13 49 27 "<IP_ADDRESS>" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:88.0) Gecko/20100101 Firefox/88.0" "201bf77b-a399-9138-b621-e291daa1ee5a" "www.m.net" "<IP_ADDRESS>:3000" outbound|3000||fpresenter.m-project.svc.cluster.local <IP_ADDRESS>:51784 <IP_ADDRESS>:8443 <IP_ADDRESS>:28398 www.m.net -

What can be the problem?

Did you apply any authorization policy? I'm trying to figure out who returned the 403 and it looks like it's returned by your backend?  (403 - via_upstream)

Well, I'll try to attach the configurations files:
Virtual service and gateway definitions:
m-vservice.yaml.txt
Request authentication:
m-ra.yaml.txt
Authorizations  for the other three services:
a-fadmin.yaml.txt
a-feditor.yaml.txt
a-fuser.yaml.txt
Deployment of the login application (fpresenter)
fpresenter.yaml.txt
What I'm trying to do:

https://www.my.net/ accessed by authenticated and not authenticated users, application fpresenter
https://www.my.net/pres accessed by authenticated  users, application fpresenter
https://www.my.net/user accessed by authenticated users with role "user", application fuser
https://www.my.net/editor accessed by authenticated users with role "editor",  application feditor
https://www.my.net/admin accessed by authenticated users with role "admin",  application fadmin

I also tried access with curl:
KC_USERNAME=martino
KC_PASSWORD=martino
KC_CLIENT_ID=m-client
KC_CLIENT_SECRET=18090b88-b8b7-4480-b4ae-469eed644333
KC_ISSUER=https://k6k.m.net/auth/realms/m-project

KC_RESPONSE=$( \
curl \
  -d "client_id=$KC_CLIENT_ID" \
  -d "client_secret=$KC_CLIENT_SECRET" \
  -d "username=$KC_USERNAME" \
  -d "password=$KC_PASSWORD" \
  -d "grant_type=password" \
  -d "scope=profile openid" \
  "$KC_ISSUER/protocol/openid-connect/token" \
)
KC_ID_TOKEN=$(echo $KC_RESPONSE | jq -r .id_token)
TOKEN=$(echo $KC_RESPONSE | jq -r .access_token)
echo $TOKEN
clear
curl --header "Authorization: Bearer $TOKEN" https://www.m.net -s -o /dev/null -w "%{http_code}\n"
curl --header "Authorization: Bearer $TOKEN" https://www.m.net/pres -s -o /dev/null -w "%{http_code}\n"
curl --header "Authorization: Bearer $TOKEN" https://www.m.net/user -s -o /dev/null -w "%{http_code}\n"
curl --header "Authorization: Bearer $TOKEN" https://www.m.net/editor -s -o /dev/null -w "%{http_code}\n"
curl --header "Authorization: Bearer $TOKEN" https://www.m.net/admin -s -o /dev/null -w "%{http_code}\n"

Obtaining these return codes:
mirto@c37-mirto:~$ curl --header "Authorization: Bearer $TOKEN" https://www.m.net -s -o /dev/null -w "%{http_code}\n"
200
mirto@c37-mirto:~$ curl --header "Authorization: Bearer $TOKEN" https://www.m.net/pres -s -o /dev/null -w "%{http_code}\n"
302
mirto@c37-mirto:~$ curl --header "Authorization: Bearer $TOKEN" https://www.m.net/user -s -o /dev/null -w "%{http_code}\n"
200
mirto@c37-mirto:~$ curl --header "Authorization: Bearer $TOKEN" https://www.m.net/editor -s -o /dev/null -w "%{http_code}\n"
403
mirto@c37-mirto:~$ curl --header "Authorization: Bearer $TOKEN" https://www.m.net/admin -s -o /dev/null -w "%{http_code}\n"
403
mirto@c37-mirto:~$

Seems something related to authorizations.

Well, to simplify the test, I removed all RequestAuthentication and AuthorizationPolicy
The application works correctly with the path: browser --> Apache reverse proxy --> docker --> application
sysop@m-serv:~$ docker run -p 3000:3000 --name fpresenter -d fpresenter:v1
c7b180ef953424f4ad818b6a98be7c264beeb86fddcab352108dec1d84d2cf76
sysop@m-serv:~$ docker logs --follow  fpresenter

><EMAIL_ADDRESS>start
> node --trace-warnings ./bin/www

GET / 200 1739.193 ms - 1244
GET /presstatic/stylesheets/style.css 200 23.220 ms - 166
GET /presstatic/images/favicon.ico 200 8.590 ms - 1150
GET /pres/secured 302 33.729 ms - 554
GET /pres/secured?auth_callback=1&state=4ce511e0-cbad-48f1-80ed-420ba0381ccf&session_state=d8387e87-0c4d-44ed-9136-33f3b3dda75b&code=4fe37ed3-557b-44e6-bed5-9927f42a34a8.d8387e87-0c4d-44ed-9136-33f3b3dda75b.e624b94f-f568-4249-9cb0-73f8c25b3955 302 83.726 ms - 70
GET /pres/secured 200 51.982 ms - 1467
GET /presstatic/stylesheets/style.css 200 1.570 ms - 166
GET /presstatic/images/favicon.ico 200 1.051 ms - 1150

Instead I have the error
Could not obtain grant code: Error: Client network socket disconnected before secure TLS connection was established

with the path browser --> istio ingressgateway --> pod --> istio sidecar --> application
npm WARN registry Unexpected warning for https://registry.npmjs.org/: Miscellaneous Warning ECONNRESET: request to https://registry.npmjs.org/npm failed, reason: Client network socket disconnected before secure TLS connection was established
npm WARN registry Using stale data from https://registry.npmjs.org/ due to a request error during revalidation.
><EMAIL_ADDRESS>start
> node --trace-warnings ./bin/www
GET / 200 1624.080 ms - 1244
GET /pres/secured 302 37.030 ms - 258
GET /presstatic/images/favicon.ico 200 44.051 ms - 1150
GET / 200 93.874 ms - 1244
GET /pres/secured 302 25.414 ms - 258
GET /presstatic/images/favicon.ico 200 1.184 ms - 1150
Could not obtain grant code: Error: Client network socket disconnected before secure TLS connection was established
GET /pres/secured?auth_callback=1&state=d648b946-27c7-42a5-9988-fb0c043aeda0&session_state=5d137aba-1f83-465c-9557-b9e2368bd193&code=73ce1472-5bcf-4337-8730-93e4ee15151c.5d137aba-1f83-465c-9557-b9e2368bd193.e624b94f-f568-4249-9cb0-73f8c25b3955 403 51.127 ms - -

I don't know what is happening

SOLVED
the problem was the istio operator installation using
    outboundTrafficPolicy:
      mode: REGISTRY_ONLY

Instead of
    outboundTrafficPolicy:
      mode: ALLOW_ANY

The correct file to install Istio is
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
  namespace: istio-system
  name: example-istiocontrolplane
spec:
  profile: demo
  meshConfig:
    accessLogEncoding: TEXT
    accessLogFile: "/dev/stdout"
    accessLogFormat: ""
    outboundTrafficPolicy:
      mode: ALLOW_ANY
  values:
    pilot:
      jwksResolverExtraRootCA: |
        -----BEGIN CERTIFICATE-----
        MIIEKTCCAxGgAwIBAgIUNjT4liTyGcGTewYUb2NZj4oWMGowDQYJKoZIhvcNAQEL
        BQAwgaMxCzAJBgNVBAYTAklUMQ4wDAYDVQQIDAVJdGFseTENMAsGA1UEBwwEUm9t
        ZTEcMBoGA1UECgwTQnVzaWNvIE1pcnRvIFNpbHZpbzETMBEGA1UECwwKTGFib3Jh
        dG9yeTEcMBoGA1UEAwwTQnVzaWNvIE1pcnRvIFNpbHZpbzEkMCIGCSqGSIb3DQEJ
        ARYVbWlydG9idXNpY29AZ21haWwuY29tMB4XDTIxMDIxOTExNDA1N1oXDTMxMDIx
        NzExNDA1N1owgaMxCzAJBgNVBAYTAklUMQ4wDAYDVQQIDAVJdGFseTENMAsGA1UE
        BwwEUm9tZTEcMBoGA1UECgwTQnVzaWNvIE1pcnRvIFNpbHZpbzETMBEGA1UECwwK
        TGFib3JhdG9yeTEcMBoGA1UEAwwTQnVzaWNvIE1pcnRvIFNpbHZpbzEkMCIGCSqG
        SIb3DQEJARYVbWlydG9idXNpY29AZ21haWwuY29tMIIBIjANBgkqhkiG9w0BAQEF
        AAOCAQ8AMIIBCgKCAQEAtt+v2C9p60rx1Q/yOQsgsis/dBNAo4efFlyN0Ibs9ts4
        a0LRQp4EwZpv9+tysVqGZvN4fJ99mdyiJHiFlchMfq4t+OzOHnym7Yi5khHS5/rv
        TwvwD+1igMny1FybVOxSlfdZGF5mhgRFD6mQod/hix5QJgegmygEhj0VV/i2rZhH
        FW0oMR1smLfALQQZhGJ//TgCjNlpK2D6zlxEXIr+QLyAje+kQyyqkJefY/Vggg9m
        gRFV7gu3MKKGE5B+ESfqhZbUqsKz/rsxs2L23Selp3FM+DhKIC8DM06dgAh7DYUQ
        IeIe9HT7+RTTs3KM7ArDXrF+BF+D8O/a4D3YBDhHswIDAQABo1MwUTAdBgNVHQ4E
        FgQUGHqUwQ6vRxvpB7+MCUEAwmUgbtAwHwYDVR0jBBgwFoAUGHqUwQ6vRxvpB7+M
        CUEAwmUgbtAwDwYDVR0TAQH/BAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEAVThR
        2NmNuey07twCW4/B8v6zOCeO/n8Z+waRw1XK9XmA+QPUTi+bLKvfx+7RVgaZD6SR
        EQHMCshGD7In5PbSBsrp6ocmCvcopd2iqvt2GLvJHuZy7hI+RgaMgQo9hhThHf9e
        FoW3C41Mm3ofvUubIKLEFKnCvxOsD+Ayyg69pGwNM+PQK1XvXWWYm8eroPICxriq
        8ULUgAc+leGiHbAKSXGLj6U/njyRzkxAmXzkierT1DFDHa9sZst7nCaSycKY7rBj
        GU2xRTOpYrQHcsaBZBjTT8/ag2IasCzFVeZ5+bMmTaDus5QT3tIubR8ukTx5jf0S
        BiQTL+/xni6Fxkrl3Q==
        -----END CERTIFICATE-----

With this the callback works correctly


decoder_cell.get_initial_state fails in eager mode

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Online, colab
TensorFlow version and how it was installed (source or binary): 2.1.0
TensorFlow-Addons version and how it was installed (source or binary): 0.8.1
Python version: 3.6
Is GPU used? (yes/no): both throw an exception

Describe the bug
decoder_cell.get_initial_state throws an exception when called.
Code to reproduce the issue
Colab notebook: https://colab.research.google.com/drive/1iwLGirQ9_avql9sKsGY4Gc5JF1rpwYrO
# Input Layer
enc_input_layer = tf.keras.Input(shape = [None], dtype = tf.int32)

# Embedding Layer
EMBEDDING_SIZE = 12
VOCAB_SIZE = 34
enc_embed_layer = tf.keras.layers.Embedding(VOCAB_SIZE, EMBEDDING_SIZE)(enc_input_layer)

# Encoder
LSTM_SIZE_ENC = 512  # num_units

encoder = tf.keras.layers.LSTM(LSTM_SIZE_ENC, return_state=True)
encoder_outputs, state_h, state_c = encoder(enc_embed_layer)
encoder_state = [state_h, state_c]

# Decoder
LSTM_SIZE_DEC = 512
ATTENTION_UNITS = 2

decoder_cell = tfa.seq2seq.AttentionWrapper(
        tf.keras.layers.LSTMCell(LSTM_SIZE_DEC),
        attention_mechanism = tfa.seq2seq.LuongAttention(
            ATTENTION_UNITS,
            #memory_sequence_length = 32
            memory = encoder_state),
        attention_layer_size = ATTENTION_UNITS)

# get the initial_state to call decoder
BATCH_SIZE = 4
initial_state = decoder_cell.get_initial_state(dtype=tf.float32, batch_size=BATCH_SIZE)

Other info / logs
---------------------------------------------------------------------------
OperatorNotAllowedInGraphError            Traceback (most recent call last)
<ipython-input-14-70f3e9fceb40> in <module>()
      2
      3 if with_attn:
----> 4     initial_state = decoder_cell.get_initial_state(dtype=tf.float32, batch_size=BATCH_SIZE)  # Crashes(!)
      5     initial_state = initial_state.clone(cell_state=encoder_state)
      6 else:

8 frames
/tensorflow-2.1.0/python3.6/tensorflow_addons/seq2seq/attention_wrapper.py in get_initial_state(self, inputs, batch_size, dtype)
   1960             )
   1961             with tf.control_dependencies(
-> 1962                 self._batch_size_checks(batch_size, error_message)
   1963             ):  # pylint: disable=bad-continuation
   1964                 cell_state = tf.nest.map_structure(

/tensorflow-2.1.0/python3.6/tensorflow_addons/seq2seq/attention_wrapper.py in _batch_size_checks(self, batch_size, error_message)
   1839                 batch_size, attention_mechanism.batch_size, message=error_message
   1840             )
-> 1841             for attention_mechanism in self._attention_mechanisms
   1842         ]
   1843

/tensorflow-2.1.0/python3.6/tensorflow_addons/seq2seq/attention_wrapper.py in <listcomp>(.0)
   1839                 batch_size, attention_mechanism.batch_size, message=error_message
   1840             )
-> 1841             for attention_mechanism in self._attention_mechanisms
   1842         ]
   1843

/tensorflow-2.1.0/python3.6/tensorflow_core/python/ops/check_ops.py in assert_equal_v2(x, y, message, summarize, name)
    645       execution or if `x` and `y` are statically known.
    646   """
--> 647   return assert_equal(x=x, y=y, summarize=summarize, message=message, name=name)
    648
    649

/tensorflow-2.1.0/python3.6/tensorflow_core/python/ops/check_ops.py in assert_equal(x, y, data, summarize, message, name)
    656       return None if context.executing_eagerly() else control_flow_ops.no_op()
    657   return _binary_assert('==', 'assert_equal', math_ops.equal, np.equal, x, y,
--> 658                         data, summarize, message, name)
    659
    660

/tensorflow-2.1.0/python3.6/tensorflow_core/python/ops/check_ops.py in _binary_assert(sym, opname, op_func, static_func, x, y, data, summarize, message, name)
    331       test_op = op_func(x, y)
    332       condition = math_ops.reduce_all(test_op)
--> 333       if condition:
    334         return
    335

/tensorflow-2.1.0/python3.6/tensorflow_core/python/framework/ops.py in __bool__(self)
    755       `TypeError`.
    756     """
--> 757     self._disallow_bool_casting()
    758
    759   def __nonzero__(self):

/tensorflow-2.1.0/python3.6/tensorflow_core/python/framework/ops.py in _disallow_bool_casting(self)
    524     else:
    525       # Default: V1-style Graph execution.
--> 526       self._disallow_in_graph_mode("using a `tf.Tensor` as a Python `bool`")
    527
    528   def _disallow_iteration(self):

/tensorflow-2.1.0/python3.6/tensorflow_core/python/framework/ops.py in _disallow_in_graph_mode(self, task)
    513     raise errors.OperatorNotAllowedInGraphError(
    514         "{} is not allowed in Graph execution. Use Eager execution or decorate"
--> 515         " this function with @tf.function.".format(task))
    516
    517   def _disallow_bool_casting(self):

OperatorNotAllowedInGraphError: using a `tf.Tensor` as a Python `bool` is not allowed in Graph execution. Use Eager execution or decorate this function with @tf.function.

disable eager
When .. is set
tf.compat.v1.disable_eager_execution()

then decoder_cell.get_initial_state works.

@guillaumekln Do you know if it's a known problem?
When I run the full code without eager mode, it ends up in another error
... [see code above] ...
# get the initial_state to call decoder
BATCH_SIZE = 4
initial_state = decoder_cell.get_initial_state(dtype=tf.float32, batch_size=BATCH_SIZE)

decoder = tfa.seq2seq.BasicDecoder(
    cell = decoder_cell,
    sampler = tfa.seq2seq.TrainingSampler(),
    output_layer = tf.keras.layers.Dense(VOCAB_SIZE)  # the projection layer with num_outputs
)

dec_input_layer = tf.keras.Input(shape = [None], dtype = tf.int32)
dec_embed_layer = tf.keras.layers.Embedding(VOCAB_SIZE, EMBEDDING_SIZE)(dec_input_layer)

outputs, _, _ = decoder(
    dec_embed_layer,
    initial_state=initial_state
)

New error
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-7-9588bb9b4f9e> in <module>()
     43 outputs, _, _ = decoder(
     44     dec_embed_layer,
---> 45     initial_state=initial_state
     46 )

1 frames
/tensorflow-2.1.0/python3.6/tensorflow_core/python/keras/engine/base_layer.py in __call__(self, inputs, *args, **kwargs)
    776                     outputs = base_layer_utils.mark_as_return(outputs, acd)
    777                 else:
--> 778                   outputs = call_fn(cast_inputs, *args, **kwargs)
    779
    780             except errors.OperatorNotAllowedInGraphError as e:

/tensorflow-2.1.0/python3.6/tensorflow_core/python/autograph/impl/api.py in wrapper(*args, **kwargs)
    235       except Exception as e:  # pylint:disable=broad-except
    236         if hasattr(e, 'ag_error_metadata'):
--> 237           raise e.ag_error_metadata.to_exception(e)
    238         else:
    239           raise

ValueError: in converted code:

    /tensorflow-2.1.0/python3.6/tensorflow_addons/seq2seq/decoder.py:162 call  *
        return dynamic_decode(
    /tensorflow-2.1.0/python3.6/tensorflow_addons/seq2seq/decoder.py:405 body  *
        (next_outputs, decoder_state, next_inputs, decoder_finished) = decoder.step(
    /tensorflow-2.1.0/python3.6/tensorflow_core/python/ops/control_flow_ops.py:2478 while_loop_v2
        return_same_structure=True)
    /tensorflow-2.1.0/python3.6/tensorflow_addons/seq2seq/basic_decoder.py:134 step  *
        cell_outputs, cell_state = self.cell(inputs, state, training=training)
    /tensorflow-2.1.0/python3.6/tensorflow_core/python/keras/engine/base_layer.py:778 __call__
        outputs = call_fn(cast_inputs, *args, **kwargs)
    /tensorflow-2.1.0/python3.6/tensorflow_addons/seq2seq/attention_wrapper.py:2065 call  *
        attention, alignments, next_attention_state = self._attention_fn(
    /tensorflow-2.1.0/python3.6/tensorflow_addons/seq2seq/attention_wrapper.py:1572 _compute_attention  *
        alignments, next_attention_state = attention_mechanism(
    /tensorflow-2.1.0/python3.6/tensorflow_addons/seq2seq/attention_wrapper.py:201 __call__  *
        return super().__call__(inputs, **kwargs)
    /tensorflow-2.1.0/python3.6/tensorflow_core/python/keras/engine/base_layer.py:778 __call__
        outputs = call_fn(cast_inputs, *args, **kwargs)
    /tensorflow-2.1.0/python3.6/tensorflow_addons/seq2seq/attention_wrapper.py:265 call  *
        return self._calculate_attention(query, state)
    /tensorflow-2.1.0/python3.6/tensorflow_addons/seq2seq/attention_wrapper.py:613 _calculate_attention  *
        score = _luong_score(query, self.keys, self.scale_weight)
    /tensorflow-2.1.0/python3.6/tensorflow_addons/seq2seq/attention_wrapper.py:483 _luong_score  *
        raise ValueError(

    ValueError: Incompatible or unknown inner dimensions between query and keys. Query (Tensor("attention_wrapper_2/checked_cell_output:0", shape=(4, 512), dtype=float32)) has units: 512.  Keys (Tensor("LuongAttention_2/LuongAttention/memory_layer/Tensordot:0", shape=(None, 512, 2), dtype=float32)) have units: 2.  Perhaps you need to set num_units to the keys' dimension (2)?

The initial error is the same issue as #673, but it was closed without a complete resolution. It is because you set the BATCH_SIZE to a Python value, while it should depend on your input, e.g. tf.shape(enc_embed_layer)[0].
But after this change, there is another error (as reported in #673). As a non Keras expert, I'm still trying to figure this out. When running Keras in eager mode there is a strange mix of eager and graph tensors.

@ulf1, please find the correct code below. It still has to be executed in the Graph mode because of some misappropriate graph tensors management in the Eager mode deep inside Tensorflow core.
import tensorflow as tf
import tensorflow.keras as keras
import tensorflow_addons as tfa
print(tf.__version__)

# Disable eager mode
tf.compat.v1.disable_eager_execution()

# Input Layer
enc_input_layer = tf.keras.Input(shape = [None], dtype = tf.int32)

# Embedding Layer
EMBEDDING_SIZE = 12
VOCAB_SIZE = 34
enc_embed_layer = tf.keras.layers.Embedding(VOCAB_SIZE, EMBEDDING_SIZE)(enc_input_layer)

# Encoder
LSTM_SIZE_ENC = 512  # num_units

encoder = tf.keras.layers.LSTM(LSTM_SIZE_ENC, return_sequences=True,return_state=True)
encoder_outputs, state_h, state_c = encoder(enc_embed_layer)
encoder_state = [state_h, state_c]

# Decoder
LSTM_SIZE_DEC = 512
ATTENTION_UNITS = 2

decoder_cell = tfa.seq2seq.AttentionWrapper(
        tf.keras.layers.LSTMCell(ATTENTION_UNITS),
        attention_mechanism = tfa.seq2seq.LuongAttention(
            ATTENTION_UNITS,
            #memory_sequence_length = 32
            memory = encoder_outputs),
        )#attention_layer_size = ATTENTION_UNITS)

# get the initial_state to call decoder
BATCH_SIZE = tf.shape(enc_embed_layer)[0]
initial_state = decoder_cell.get_initial_state(dtype=tf.float32, batch_size=BATCH_SIZE)

# Instantiate Decoder
decoder = tfa.seq2seq.BasicDecoder(
    cell = decoder_cell,
    sampler = tfa.seq2seq.TrainingSampler(),
    output_layer = tf.keras.layers.Dense(VOCAB_SIZE)  # the projection layer with num_outputs
)

# Input Layer
dec_input_layer = tf.keras.Input(shape = [None], dtype = tf.int32)
# Embedding Layer
dec_embed_layer = tf.keras.layers.Embedding(VOCAB_SIZE, EMBEDDING_SIZE)(dec_input_layer)

outputs, _, _ = decoder(
    dec_embed_layer,
    initial_state=initial_state
)

# logits = outputs.rnn_output

@failure-to-thrive Thank you

So we need to wait for tensorflow core to change.

So they claim they fixed it: https://github.com/tensorflow/tensorflow/issues/37264#issuecomment-596834233


Scrolling is not working in the viewer

Text highlighting and scrolling is not working

Ok - I found the code that's been removed but there's an issue. Given that an item can have both a transcription and interlinear text I can't actually tell which one to scroll. We need to think of a way to show only one or the other (either the transcription or the interlinear) and enable scrolling them.

@nthieberger Any ideas on how to tackle this?
The issue stems from the fact that there can be multiple transcriptions on the page at once. Playing the media then results in a situation where the app doesn't know which transcription to scroll.
Perhaps it should be rejigged to show only the list of transcriptions available (without actually drawing them). The user would then need to select one to display (and only be able to show one at a time) that would be scrolled as the media plays.
Thoughts?

Yes, showing only one is fine, and having it close when another is selected is ok too

New version with a selector to choose which to display pushed to testing. See http://catalog.paradisec.org.au/test-viewer/#/NT1/98007/NT1-98007-98007A


Initialize Demo App

Summary
The demo app for this project.
Todo

[ ] Init vite project
[ ] Implement pages and modules
[ ] figure out the model post processing part in js
[ ] setup CI/CD

Wonderful work！ Looking forward to model training guideline.


Extension does not work with process template inherited from out-of-the-box Agile process template

I installed version 1.0.6 of your extension on Dec 19, 2020 at 8 pm EST on Azure DevOps Server 2019, Update 1.1, patch 6
Here is my json file:
{
  "tasks": [
    {
      "name": "Design Task",
      "fields": [
        {
          "name": "System.Title",
          "value": "{System.Title} Design FOO"
        }
      ]
    },
    {
      "name": "Dev Task",
      "fields": [
        {
          "name": "System.Title",
          "value": "{System.Title} Dev FOO"
        }
      ]
    },
    {
      "name": "Test Task",
      "fields": [
        {
          "name": "System.Title",
          "value": "{System.Title} Test FOO"
        }
      ]
    }
  ]
}

For a project based on the out-of-the-box Microsoft Agile process template, the template works great
For a project based on a custom process template that I inherited from the out-of-the-box Microsoft Agile process template, it does not work. Note: I created the customized process template using inheritance from the web portal and NOT by exporting/editing XML. Everything works as its supposed to except: When I'm editing a user story and click the Add tasks button, nothing happens, and the Edge console shows the following (see screen shot).
If you want, I'll capture logs and send them to you. Let me know

Hi,
I can reproduce the error now. It seems that the issue comes from Kanban columns fields. I added a try/catch on this error to log on console instead of throwing an uncatched exception.

Created task 10911 ChildTasksService.ts:101:16
An error accured while setting field value. Name 'WEF_87E142FB9098434680E3C40F48B1A790_Kanban.Column.Done'; Value 'false'.can't assign to property "Done" on "Waiting user response": not an object ChildTasksService.ts:123:16
An error accured while setting field value. Name 'WEF_259A92C5D3BE40C28675E8E6A87AA6AD_Kanban.Column.Done'; Value 'false'.can't assign to property "Done" on "Active": not an object ChildTasksService.ts:123:16
An error accured while setting field value. Name 'WEF_87E142FB9098434680E3C40F48B1A790_Kanban.Column.Done'; Value 'false'.can't assign to property "Done" on "Waiting user response": not an object ChildTasksService.ts:123:16
An error accured while setting field value. Name 'WEF_259A92C5D3BE40C28675E8E6A87AA6AD_Kanban.Column.Done'; Value 'false'.can't assign to property "Done" on "Active": not an object ChildTasksService.ts:123:16
An error accured while setting field value. Name 'WEF_87E142FB9098434680E3C40F48B1A790_Kanban.Column.Done'; Value 'false'.can't assign to property "Done" on "Waiting user response": not an object ChildTasksService.ts:123:16
An error accured while setting field value. Name 'WEF_259A92C5D3BE40C28675E8E6A87AA6AD_Kanban.Column.Done'; Value 'false'.can't assign to property "Done" on "Active": not an object ChildTasksService.ts:123:16
An error accured while setting field value. Name 'WEF_87E142FB9098434680E3C40F48B1A790_Kanban.Column.Done'; Value 'false'.can't assign to property "Done" on "Waiting user response": not an object ChildTasksService.ts:123:16
An error accured while setting field value. Name 'WEF_259A92C5D3BE40C28675E8E6A87AA6AD_Kanban.Column.Done'; Value 'false'.can't assign to property "Done" on "Active": not an object ChildTasksService.ts:123:16
Created task 10912

Hi, this has been fixed in version 1.0.7 (april 11)


feat(derive): batch stream provider

#591  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @refcell and the rest of your teammates on  Graphite


Add support for IO body (for websockets)

https://github.com/http-rs/tide/issues/67
https://github.com/http-rs/http-types/pull/231

closing this — upgrades are in async-h1 2.2.0


unable to replicate results for cifar100

When training efficientnet-b0 on Cifar 100, while the reported results are ~90% accuracy, the model achieves an accuracy of around 7% (Cifar 10 achieves around 40% accuracy).
I've explored several optimizers  (including SGD and RMSprop) with several learning rate schedulers, none seem to affect (used https://github.com/katsura-jp/efficientnet-pytorch/blob/master/cifar100.py as reference).
Any acute modifications has to be made (other than changing the input size and classes)?
additional information:
keras version: 2.2.4
TF version: 1.14.0
Data scaling: [-0.5,0.5]
Thanks

@shairoz The dataset you are using has 100 classes and total training samples are 50000 which approximately makes 500 class per-samples. Due to the above reason this model would not perfectly generalize your dataset therefore it is recommended to use a simpler model for training.

The reason for the reported accuracy to be ~90% of EfficientNet-B0 on CIFAR-100 is that the authors have used transfer learning.
May be by initializing your initial weights with the available pre-trained weights help you.


Node fails to connect to Go GRPC: http1.x

My node process is unable to connect to a GRPC Golang server on OS X. I believe the source code is fine because it worked last week, but this is a new node installation so I was wondering what component may be broken / related? This does work fine in a Docker container, too.
This may be a wider issue though... maybe a dep of Node? Maybe OpenSSL (since HTTP/2 requires TLS?)? I see a similar issue here- https://github.com/grpc/grpc/issues/7726. I've tried the stupid stuff like npm rebuild and rm -r node_modules && npm i
{
  "message": "{\"created\":\"@1492321122.263552000\",\"description\":\"Failed parsing HTTP/2\",\"file\":\"../src/core/ext/transport/chttp2/transport/chttp2_transport.c\",\"file_line\":1869,\"grpc_status\":14,\"referenced_errors\":[{\"created\":\"@1492321122.263541000\",\"description\":\"Expected SETTINGS frame as the first frame, got frame type 80\",\"file\":\"../src/core/ext/transport/chttp2/transport/parsing.c\",\"file_line\":280},{\"created\":\"@1492321122.263547000\",\"description\":\"Trying to connect an http1.x server\",\"file\":\"../src/core/ext/transport/chttp2/transport/chttp2_transport.c\",\"file_line\":1830,\"http_status\":400}]}",
  "name": "Error",
  "stack": "Error: {\"created\":\"@1492321122.263552000\",\"description\":\"Failed parsing HTTP/2\",\"file\":\"../src/core/ext/transport/chttp2/transport/chttp2_transport.c\",\"file_line\":1869,\"grpc_status\":14,\"referenced_errors\":[{\"created\":\"@1492321122.263541000\",\"description\":\"Expected SETTINGS frame as the first frame, got frame type 80\",\"file\":\"../src/core/ext/transport/chttp2/transport/parsing.c\",\"file_line\":280},{\"created\":\"@1492321122.263547000\",\"description\":\"Trying to connect an http1.x server\",\"file\":\"../src/core/ext/transport/chttp2/transport/chttp2_transport.c\",\"file_line\":1830,\"http_status\":400}]}\n    at /Users/tejas/dev/src/github.com/segment-sources/stripe/node_modules/grpc/src/node/src/client.js:434:17",
  "code": 14
}

Node client code
const proto = grpc.load(`${__dirname}/../domain/library.proto`)
const credentials = grpc.credentials.createInsecure()
const libraryServiceURL = process.env['LIBRARY_SERVICE_URL'] || 'localhost:4000'
const libraryService = new proto.domain.LibraryService(libraryServiceURL, credentials)

Promise.promisifyAll(libraryService, { suffix: 'AsPromised' })

async function index(ctx) {
  const resp = await libraryService.getCatalogAsPromised({})
  ctx.render('index', { libraries: resp.libraries })
}

Go server code
	server := grpc.NewServer()
	service := Service{Catalog: fakeCatalog}
	domain.RegisterLibraryServiceServer(server, &service)

	sock, err := net.Listen("tcp", ":4000")
	if err != nil {
		log.Fatalf("failed to listen: %v", err)
	}
	server.Serve(sock)

I have a hunch this has to do with mismatched GRPC versions on client and server. I know I'm using node's grpc module at 1.1.2, but any idea how to find out what protocol version that is and how it compared to grpc-go? grpc-go's releases don't line up with grpc node :(

It's weird though. Apps that used to work stopped working with the same node_modules (no changes there)... maybe, something to do with node installation itself?  I got some more details on the error from the stack in another node GRPC program.
{
  "message": "{\"created\":\"@1492321122.263552000\",\"description\":\"Failed parsing HTTP/2\",\"file\":\"../src/core/ext/transport/chttp2/transport/chttp2_transport.c\",\"file_line\":1869,\"grpc_status\":14,\"referenced_errors\":[{\"created\":\"@1492321122.263541000\",\"description\":\"Expected SETTINGS frame as the first frame, got frame type 80\",\"file\":\"../src/core/ext/transport/chttp2/transport/parsing.c\",\"file_line\":280},{\"created\":\"@1492321122.263547000\",\"description\":\"Trying to connect an http1.x server\",\"file\":\"../src/core/ext/transport/chttp2/transport/chttp2_transport.c\",\"file_line\":1830,\"http_status\":400}]}",
  "name": "Error",
  "stack": "Error: {\"created\":\"@1492321122.263552000\",\"description\":\"Failed parsing HTTP/2\",\"file\":\"../src/core/ext/transport/chttp2/transport/chttp2_transport.c\",\"file_line\":1869,\"grpc_status\":14,\"referenced_errors\":[{\"created\":\"@1492321122.263541000\",\"description\":\"Expected SETTINGS frame as the first frame, got frame type 80\",\"file\":\"../src/core/ext/transport/chttp2/transport/parsing.c\",\"file_line\":280},{\"created\":\"@1492321122.263547000\",\"description\":\"Trying to connect an http1.x server\",\"file\":\"../src/core/ext/transport/chttp2/transport/chttp2_transport.c\",\"file_line\":1830,\"http_status\":400}]}\n    at /Users/tejas/dev/src/github.com/segment-sources/stripe/node_modules/grpc/src/node/src/client.js:434:17",
  "code": 14
}

Oh man, this actually seems to have nothing to do with the version of node_modules. I spun up a Docker container (linux) and re-npm-install'd inside of it (already tried that on my host-- OS X) and am unable to reproduce... so it must be related to my Node.js setup locally.
This may be a wider issue though... maybe a dep of Node? Maybe OpenSSL (since HTTP/2 requires TLS?)? I see a similar issue here- https://github.com/grpc/grpc/issues/7726

The Node code looks right, though I'm not entirely familiar with the ES6 idioms. The error says (if you scroll far enough) "Trying to connect an http1.x server". Is it possible that the environment variable LIBRARY_SERVICE_URL is set, and you're not actually connecting to the local server? Are you sure that the Go server is successfully listening on that port, and that some other HTTP 1 server isn't already listening there?

@murgatroid99 Nah, env var isn't set. Happens for all GRPC Node apps in my host environment. It's quite a peculiar issue... source code works fine in Docker container. Happy to dive deeper... just a bit stuck, let me know if you have any debugging tips

If it's happening to every gRPC Node app in your host environment, the simplest explanation is that they are actually connecting through a proxy, as was mentioned in the issue you linked. In general, to get a lot of debugging information, you can run the Node client with the environment variables GRPC_TRACE=all GRPC_VERBOSITY=DEBUG. A dump of that log from a run of the client could help find the source of the problem. If you decide to share it, check that it doesn't have anything private.

I am experiencing a similar issue.  The error reproduces on Ubuntu 14 as well (node client / golang server).
D0613 21:16:33.311846000<PHONE_NUMBER>06112 connectivity_state.c:185]     SET: 0x10201c3b0 (null): READY --> SHUTDOWN [selected_changed] error=0x1020a2120 {"created":"@1497413793.310465000","description":"Delayed close due to in-progress write","file":"../src/core/ext/transport/chttp2/transport/chttp2_transport.c","file_line":591,"referenced_errors":[{"created":"@1497413793.310464000","description":"Failed parsing HTTP/2","file":"../src/core/ext/transport/chttp2/transport/chttp2_transport.c","file_line":2166,"grpc_status":14,"referenced_errors":[{"created":"@1497413793.310461000","description":"frame of size 102 overflows incoming window of 1","file":"../src/core/ext/transport/chttp2/transport/parsing.c","file_line":413}]}]}

There was an integer overflow bug in flow control logic that should be already fixed in HEAD in the Go implementation. Can you please try this with the Go HEAD and see if the issue exists? Otherwise it is very likely that you are encountering the same issue.

Thank you @mehrdada for your rapid response. It would appear that the issue does not persist after upgrading to Go HEAD. I suppose this is a little obvious in hind sight.
Thank you for your help!

I'm going to close this issue as it is a duplicate of grpc/grpc-go#1192. If you are facing this, please make sure you upgrade your Go side to HEAD and please do let us know if similar issues persist despite Go endpoints being updated. Thanks!


Assign the clear browser cache to clearCache

While reading over the code, as I'm interested in something similar, and I noticed the clear browser question was being assigned to the hideCursor variable instead of the clearCache.

I can't believe I missed this! Thanks so much, @graywolf336 :)


Review and iterate dashboard design

Describe the task
Review the current content and design of the dashboard.
Acceptance Criteria

[x] Review content and design
[x] Review previous notes to see if there has been feedback
[x] Talk to Ops Team to get feedback
[x] Create wireframe based on design updates

Additional context

Relates in part to issue #490

Hi @suhafa - as discussed today, I've created this ticket to review the design of the CIF dashboard. Please feel free to update as needed.

Suggestions to explore

Embedding Metabase questions in the application
Calendar-view to track upcoming reports

Sprint planning (June 10): Things to consider:

Matthieu mentioned including imbedding Metabase queries into the dashboard?
Calendar showing upcoming due dates to help with Ops Team planning?

Wireframe with documentation

@dleard Thanks Dylan! @LindsayMacfarlane I've created ticket #857 to capture the work done here. We can close this c:

Thank you so much @suhafa!


Paste as Testbench -- Generics should turn into constant declarations

When creating a testbench for an entity with generics, these should be turned into constants in the testbench so that it's compileable without anything further written.

Well, generics don't have to have default values so this might not work out as nicely as one would hope.

Okay, seems to work with and without a default declaration.  The testbench puts in a default field  for any generic turned constant that doesn't have a value.


Mobile App

Hey. I’m considering creating a mobile app, probably React Native, for integrating with meemo. Would that be of interest and/or is it being worked on already?

It is not being worked on at all, but would be awesome to have. So far I am falling back myself to just using a link on my Android homescreen.

Has this been done?

No this wasn't even been looked into.

Subscribed!