"Error handling message: Unknown error" when accessing "Recordings"

System Health

version
core-2021.2.3

installation_type
Home Assistant Container

dev
false

hassio
false

docker
true

virtualenv
false

python_version
3.8.7

os_name
Linux

os_version
4.19.0-14-amd64

arch
x86_64

timezone
Pacific/Auckland

Version of the custom_component:
v1.0.3
Configuration

mqtt:
  host: <IP_ADDRESS>
  port: 1883
  topic_prefix: homeassitant/frigate
  user: "xxxxxxx"
  password: "xxxxxxxx"
  stats_interval: 60

clips:
  max_seconds: 300
  tmpfs_cache_size: 950m
  retain:
    default: 7
    objects:
      person: 14

detectors:
  cpu1:
    type: cpu
  # coral:
  #   type: edgetpu
  #   device: usb

ffmpeg:
  global_args: -hide_banner -loglevel info

cameras:

  lr_wyze_tvstand:
    ffmpeg:
      inputs:
        - path: rtsp://dafang:dafang@<IP_ADDRESS>:8554/unicast
          roles:
            - detect
            - rtmp
            - clips
            - record
    record:
      enabled: True
      retain_days: 7
    width: 1920
    height: 1080
    clips:
      enabled: True
      pre_capture: 5
      post_capture: 5
    rtmp:
      enabled: True
    snapshots:
      enabled: True
      timestamp: True
      bounding_box: True
      crop: True
      retain:
        default: 3

  ms_wyze_desk:
    ffmpeg:
      inputs:
        - path: rtsp://dafang:dafang@<IP_ADDRESS>:8554/unicast
          roles:
            - detect
            - rtmp
            - clips
            - record
    record:
      enabled: True
      retain_days: 7
    width: 1920
    height: 1080
    clips:
      enabled: True
      pre_capture: 5
      post_capture: 5
    rtmp:
      enabled: True
    snapshots:
      enabled: True
      timestamp: True
      bounding_box: False
      crop: True
      retain:
        default: 3

Describe the bug
Frigate recordings will not open. Clips opens and play with no issues. Media browser displays:
Media Browsing Error
Unknown error

Debug log
[139724818409600] Error handling message: Unknown error

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/decorators.py", line 18, in _handle_async_response
    await func(hass, connection, msg)
  File "/usr/src/homeassistant/homeassistant/components/media_source/__init__.py", line 94, in websocket_browse_media
    media = await async_browse_media(hass, msg.get("media_content_id"))
  File "/usr/src/homeassistant/homeassistant/components/media_source/__init__.py", line 73, in async_browse_media
    return await _get_media_item(hass, media_content_id).async_browse()
  File "/usr/src/homeassistant/homeassistant/components/media_source/models.py", line 78, in async_browse
    return await self.async_media_source().async_browse_media(self)
  File "/config/custom_components/frigate/media_source.py", line 162, in async_browse_media
    return self._browse_recording_folders(identifier, folders)
  File "/config/custom_components/frigate/media_source.py", line 542, in _browse_recording_folders
    children=[
  File "/config/custom_components/frigate/media_source.py", line 549, in <listcomp>
    title=self._generate_recording_title(identifier, folder),
  File "/config/custom_components/frigate/media_source.py", line 529, in _generate_recording_title
    return dt.datetime.strptime(f"{folder['name']}", '%Y-%m').strftime('%B %Y')
  File "/usr/local/lib/python3.8/_strptime.py", line 568, in _strptime_datetime
    tt, fraction, gmtoff_fraction = _strptime(data_string, format)
  File "/usr/local/lib/python3.8/_strptime.py", line 349, in _strptime
    raise ValueError("time data %r does not match format %r" %
ValueError: time data '@eaDir' does not match format '%Y-%m'

Howdy,
I just ran into this trying to do a few things with the directory organization within frigate.  As far as I can tell this ended up happening because the first directory under recordings/ needs to match %Y-%m, a four-digit year and a two-digit month.  Based off the last few lines in your error message, I'd wager this is why you're seeing the error.
Try making sure all the directories underneath your recordings/ directory match the format 2021-04

Howdy,
I just ran into this trying to do a few things with the directory organization within frigate. As far as I can tell this ended up happening because the first directory under recordings/ needs to match %Y-%m, a four-digit year and a two-digit month. Based off the last few lines in your error message, I'd wager this is why you're seeing the error.
Try making sure all the directories underneath your recordings/ directory match the format 2021-04

Hey,
This is an NFS mounted share share from a Synology box. That same box has AFP running which auto generates this directory everywhere. It seems if that is the case, I can’t use the storage for both Frigate and have AFP enabled.
Thank you for the insight, at least I understand why I am getting the error a little more clearly now.


BUG fix routing for react sections

Merge together with https://github.com/silverstripe/silverstripe-asset-admin/pull/116
There is an interesting and still to be improved issue when the top level CMS section is changing; Registered routes which handle sub-sections within the CMS will need to suppress ajax loading in cases where they are active, otherwise LeftAndMain.js will cause a reload (via ajax).
However if they are not active, suppressing these routes will cause no reload at all, and only the url will appear to change.
In order to detect boundary navigation, I've temporarily put in a hack to detect requests from loadPanel, and use the referer to determine if the current route handler should delegate to the top level handler or not.
This working horrible hack should be replaced with a global top level "current section" state, which can only be set by LeftAndMain, but can be read by any route handler. Ideally this should be implemented when refactoring the top level route handling out of leftandmain and into a separate class.

:+1: from me

I've now removed routing from silverstripe-component. See https://github.com/silverstripe/silverstripe-asset-admin/pull/116 for the associate fix to asset-admin.

Will rebase once https://github.com/silverstripe/silverstripe-framework/pull/5346 is merged.

Oops, I didn't update the tests. It's fixed though by now. ;D


fix: ignores mouse enter events for non-actionable items (i.e. non-in…

…teractive or not active)
Fixes https://github.com/godot-escoria/escoria-issues/issues/191

Tested successfully by setting say_long_left in room 15 to inactive

Thanks !


Unable to load all of the online models

I'm stuck at the Fetching list of online models, please wait..., and somehow there is no English-Chinese language pair. I don't know whether it is in the unloaded models. Can someone help? Thanks

Hello
Tick "multilingual modals"

Got it! Thanks! :D

sorry to bother you again. After downloading the model and successfully installing it, it shows an error in my Trados 2021. How can I fix it? TT

hey. guys. why does it show No Connection? How can I fix it?

Hi,
Do you have OPUS-CAT MT Engine running on your computer when you try to use the plugin in Trados?

yeah. I think so. I left this window open like this while using Trados, right?

Yes, that should work. One thing to check is that the net.tcp port in the Settings tab of the MT engine is 8477:

If the port is 8477 and it still doesn't work, something might be blocking the net.tcp connection. There is a newer version of the plugin available, which uses a different way of communicating with the MT engine, so you could also try that. It's included in this package, under the path tradosPlugin/Development/2021: https://github.com/Helsinki-NLP/OPUS-CAT/releases/tag/opuscatcollection_v1.0.0
That newer plugin uses the HTTP port of the MT engine, so you need to make sure that both the plugin and the engine use the same port (both the plugin and engine ports can be changed in their settings).


Warning: Prop className did not match. Server.... Typography component, prop based styles

react-dom.development.js?61bb:530 Warning: Prop className did not match.
Server: "MuiTypography-root makeStyles-root-32 makeStyles-root-47 makeStyles-darkest-27 makeStyles-bodySmall-43 makeStyles-noTextTransform-41 MuiTypography-body1"
Client: "MuiTypography-root makeStyles-root-32 makeStyles-root-48 makeStyles-darkest-27 makeStyles-bodySmall-43 makeStyles-noTextTransform-41 MuiTypography-body1"
I have tried the following so far to no avail:

configured runtime chunk in webpack, as we are code splitting at the route level,

optimization: {
    runtimeChunk: {
      name: 'app',
    },
  },

verified there is only one version of material-ui, the latest, 4.9.3

node environment is the same for both client and server

tried wrapping both the client and app in stylesprovider with a fresh createGenerateClassName:

server:
const sheets = new ServerStyleSheets();

      const generateClassName = createGenerateClassName({
        productionPrefix: 'tock',
      });

      const html = ReactDomServer.renderToString(
        sheets.collect(
          <Provider store={store}>
            <StaticRouter location={req.url} context={routerContext}>
              <StylesProvider generateClassName={generateClassName}>
                <Application />
              </StylesProvider>
            </StaticRouter>
          </Provider>
        )
      );

as well as ensuring that new ServerStyleSheets is created on every requuest.
client:
const generateClassName = createGenerateClassName({
    productionPrefix: 'tock',
  });
  try {
    (fullRender ? ReactDOM.render : ReactDOM.hydrate)(
      <StrictMode>
        <Provider store={store}>
          <Router history={tockHistory}>
            <StylesProvider generateClassName={generateClassName}>
              <Routes />
            </StylesProvider>
          </Router>
        </Provider>
      </StrictMode>,
      document.querySelector('#Root')
    );
  } catch (e) {
    // eslint-disable-next-line no-console
    console.error(e);
    throw e;
  }

followed the reference implementation:
https://material-ui.com/guides/server-rendering/
this seems to only be happening in development mode. If I bundle for production and run the app, it's working.
worked through:
https://material-ui.com/getting-started/faq/#my-app-doesnt-render-correctly-on-the-server
specifically, I was interested in this:
https://material-ui.com/getting-started/faq/#react-class-name-hydration-mismatch
Node: v13.8.0
Webpack: 4.41.6
Material-ui: 4.9.3
the component where we started seeing this issue, interestingly, uses some prop based styles:
export const useTypographyStyles = makeStyles((theme) => ({
  root: {
    marginBottom: ({ mb, paragraph }: TypographyProps) =>
      paragraph ? theme.spacing(2) : theme.spacing(mb ?? 0),
  },

which I notice generates 2 strings for the classes in the hook.
root: "makeStyles-root-32 makeStyles-root-232"

is what I get when I log the result of:
const classes = useTypographyStyles({ mb, paragraph });

but if I do a regular style there with just a value, I do not get the issue.

i'm going to try setting up code splitting in the reference implementation with my same typography wrapper.

posted in Stack overflow as well then:
https://stackoverflow.com/questions/60289525/react-dom-development-js61bb530-warning-prop-classname-did-not-match

none of the existing stack overflow answers are recent enough from what I can tell, and neither are the issues in this repo. any help would be much appreciated.

I tried it with the combination of withStyles and createStyles as well, and still have the same result.

could it be something related to compiling our server code with webpack?

ok i managed to reproduce on the reference example.
const path = require('path');
const nodeExternals = require('webpack-node-externals');
const webpack = require('webpack');

module.exports = {
  entry: './server',
  target: 'node',
  externals: [nodeExternals()],
  mode: process.env.NODE_ENV || 'development',
  output: {
    path: path.resolve(__dirname, 'build'),
    filename: 'server.js',
    chunkFilename: '[name].bundle.js',
    publicPath: '/build/',
  },
  module: {
    rules: [
      {
        test: /\.js$/,
        exclude: /node_modules/,
        loader: 'babel-loader',
      },
    ],
  },
  plugins: [
    new webpack.DefinePlugin({
      __isBrowser__: 'false',
    }),
  ],
};

compiled with webpack.
@oliviertassinari  thoughts?

will push up my repro.

I basically compiled server, and used existing webpack config for client.
then ran node build/server.js. instead of the way it's running now.

spoke too soon, tried out node polyfill settings, and when I tried it again it didn't show the error.

only way i've been able to solve this for now is to not use props in styles on SSR.

https://github.com/mui-org/material-ui/issues/18018


Hope add Stroke Position fix

Can you add some staff like Stroke Position setting? I use Sketch for design UI, it has function like that:
https://hz.ued.cat/storage/app/media/sc_20190425152412.png
Thanks!

Now try to use offset and exclude, create a fake Stroke as Insider border and outside border :P

Well, the canvas use center stroke by default but provide no parameter to control it. But we can fake the result other than offset the path, because the stroke algorithm provided by the canvas api should be more efficient and robust.
The trick here is double the strokeWidth.
For any path, to make a outer stroke, double the strokeWidth, and draw the path again without stroke, the composition is a path with outer stroke.
To make a inner stroke, double the strokeWidth, and make a clip, and the result will be a inner stroke.
And here is the way to acheive this by paper.js:
view.center = [0, 0]

let p1 = new Path.Rectangle({
    position: [0, 0],
    size: [100, 100],
    strokeWidth: 20,
    strokeColor: 'black',
    fillColor: 'red'
})

let p2 = p1.clone()
p2.translate(-200, 0)
let p3 = p1.clone()
p3.translate(200, 0)

// inner stroke
let pInner = new paper.Group({ children: [p2, p2.clone()] })
pInner.clipped = true

// center stroke
p1.strokeWidth = 10

// outer stroke
let p3Fill = p3.clone()
p3Fill.strokeWidth = 0
let pOuter = new paper.Group({ children: [p3, p3Fill] })

For size and performance concern, both paper.js and this extend library cannot make these algorithms too perfect like some native application, so the algorithms will always fail in some edge cases. But the api is implement natively and do not have these concern, their implementions should be much more robust and perfect.
So, in my opinion, if you can acheive these effects natively, do it natively.

Since the solution is offered and this function is beyond the scope of this utility, I'll close the issue. @emengweb


ServiceWorker blocked

Platform
Which platform does the bug apply to?

WebExtension (Google Chrome 81.0.4044.129, Mozilla Firefox)
NPM
Bookmarklet
https://www.birdugun.com/ https://www.manaa.io/

Describe the bug
I am using Service Worker and caching some sites. But this extension blocked my service workers. I have this error in my Console Log. I disabled wappalyzer. Everything working normal. I want to use this extension but this is so painful
To Reproduce
Steps to reproduce the behavior:

Go to 'https://www.birdugun.com/ https://www.manaa.io/'
Click on 'F12' Developer Tools, Open Console
Scroll down to '...'
See error

Expected behavior
Should not block Service Worker
Additional context
Console Error:
serviceworker:26 Uncaught (in promise) TypeError: Request scheme 'chrome-extension' is unsupported
at serviceworker:26

I believe his has been resolved, please let me know if it's still an issue.

i tested. fixed issue. thanks for helping.


Display commits on push event

This add option to display commits on push event. it can be further improved with setting flags on whether to show commits or not.

https://github.com/act10ns/slack/issues/194

Thanks. Can you add a screenshot to the PR description of what this looks like?

Thanks. Can you add a screenshot to the PR description of what this looks like?

truth be told I am not sure how to test this changes without being deployed. maybe you can guide me then I can test it and attach screenshoot

I think you'd need to add the action from your fork to a workflow in another repo and trigger a push event. Not easy now I think about it. Leave it with me. Thanks again for the PR.

thanks. the idea is basically showing the commits just below the title then show the build steps after that. the commit list should appear only on push event.

I think if you change this line. It would display the commit message.
https://github.com/act10ns/slack/blob/e4e71685b9b239384b0f676a63c32367f59c2522/src/slack.ts#L83

I think if you change this line. It would display the commit message.
https://github.com/act10ns/slack/blob/e4e71685b9b239384b0f676a63c32367f59c2522/src/slack.ts#L83

that's was the idea but I just thought of listing those commits in there as there might be more than one commit depending on people strategy on how the merge. But I think it should still be fine to not included it on the title and display commit list. Its more flexible to list them.

@satterly I can test this PR if it was on your branch for this repo, then will past screen shot for results. maybe can you merge it on a separate repo and I will test that repo on my actions. eg. uses: act10ns/slack@show-commits

@norbybaru does it working. I mean did you test it. I tested it. But it is still showing the number of commits.

@norbybaru does it working. I mean did you test it. I tested it. But it is still showing the number of commits.

@sroopsai how did you test it? I am struggling to test it on my repo. I relied on test case to test it.
The commit should only display for push event. Can you send me steps on how to test it please. I am new with Github action development

Yes, I tested by forking this repo. And used that repo in one of the workflows. I will try to test your repo now. And see whether it's working or not.

@norbybaru I tested your solution. I am inserting a picture. Please check it out.

 I am inserting a picture. Please check it out.

@sroopsai  that looks more like it.
You will see it commit hash and message just below the title. if you had more than 1 commit then they would have been listed there but ideally one shouldn't but like I said that depends on individual strategy.
The screenshot above is definitely accurate from m implementation

maybe the title could also just display no. of commit without commit has? eg. 1 commit display commit hash and title below it like in the screenshot

@satterly @sroopsai any update on this one? any changes that you not happy with that I should make changes to?

This would be awesome because right now its hard to know what happened if used following the merge of a pull request into the trunk.

However, I am not quite sure this would solve my problem cause the workflow is executing following the CI workflow using workflow_run trigger.

Any update on this?

Thanks for the PR but I'm closing this in favour of a templating approach. See #208


Feat: support for react-navigation@5 (closes #17)

references:

https://reactnavigation.org/docs/navigation-prop#isfocused---query-the-focused-state-of-the-screen
https://reactnavigation.org/docs/upgrading-from-4.x#higher-order-components
https://reactnavigation.org/docs/navigation-events/

It drops the support for v4 though

@JPeer264 thanks for the PR. I'd love to add support for version 5, but I'd prefer for that to be done using the hooks exported from react-navigation rather than adding dependency on @react-navigation/compat would you be interested in investigating that? Thank you!

Ah I already wanted to refactor it to a function component. Sure I will change it to hooks :)

If it's helpful, this is my update:
It exports a hook useAndroidBackHandler and a component <AndroidBackHandler>
import { useCallback } from 'react';
import { BackHandler } from 'react-native';
import { useFocusEffect } from '@react-navigation/native';

export function useAndroidBackHandler(onBackPress) {
  useFocusEffect(
    useCallback(() => {
      BackHandler.addEventListener('hardwareBackPress', onBackPress);

      return () =>
        BackHandler.removeEventListener('hardwareBackPress', onBackPress);
    }, [onBackPress]),
  );
}

export default function AndroidBackHandler({ onBackPress, children }) {
  useAndroidBackHandler(onBackPress);
  return children || null;
}

@euharrison I think your solution to keep the component is pretty nice 👍

Alright, I updated to hooks and tested them. I updated the README as well.
For the release notes: Upgrading to this should be seamless (if the app is already on rn-navigation@5)

@vonovak ping

@JPeer264 hello and thanks, this looks good to me. Recently, I merged this PR https://github.com/vonovak/react-navigation-backhandler/pull/16 which I think is not covered in this code. Would it possible to support that too, using https://reactnavigation.org/docs/use-is-focused/ ? Thanks!

Sure thing, I will implement it as well

@vonovak I checked the documentation of react-navigation and it says:

The effect will run whenever the dependencies passed to React.useCallback change, i.e. it'll run on initial render (if the screen is focused) as well as on subsequent renders if the dependencies have changed.

Which means #16 is covered within this PR.

thank you for the PR!


Mavlink messages do not get relayed to GCS

It seems when sending a mavlink statustext message from a board computer PX4 does not relay the message to the ground station. The question is if this should be implemented and how.
Considerations would be the use of a target_system and default behavior when such an ID is omitted.
In dronekit i tried:
msg = vehicle.message_factory.statustext_encode(6,"Test message")
vehicle.send_mavlink(msg)

I also tried
vehicle._master.mav.statustext_send(6,"Test message")

Neither works

Did you try setting the forwarding flag -f on the mavlink instance that has telemetry attached?

The is no active management on that side, it's just relaying serial to UDP

I'm not sure what you are saying. How does the setup look?

I'll pm you

Closing as devcall discussion indicated that this is not an issue.


iOS 15.5 Detents Not Working

Unless I set the Detent to fullscreen, it opens in medium detent, the Anchor Detent and Content Detent functionalities do not work on iOS 15.5 devices.

Custom detents on iOS are only supported from iOS 16 and up: https://developer.apple.com/documentation/uikit/uisheetpresentationcontroller/detent/3976719-custom
On iOS 15 and down, the only detents supported are medium and large, which translate to a RatioDetent with 0.5 and a FulScreenDetent
This is also mentioned in the readme


Create Meta Spatial Anchor tutorial doc

Adds a tutorial document to introduce users to using spatial anchors with the OpenXRFbSpatialAnchorManager node. I've also added a link at the top of the Meta Scene Manager tutorial cross referencing the two documents, as the scene / spatial anchor APIs are often used together.
This PR will probably fail CI until after #170 is merged.

Rebased so this PR will be merge ready after #187 is merged


Simple PV slicing tool

@ChrisBeaumont @jegpeek @perrygreenfield - just FYI I created this repo as a temporary sandbox to try and write plugins/viewers/tools for glue as simply as possible in a way that requires minimal understanding of the glue code base. This is meant to be a way to discuss possible APIs that users could use.
This first example shows how one might more simply define a PV slicing tool that has the same functionality as the built-in one, but would be simpler for users to write from scratch. Of course, this is not at all functional at this point, and it's not complete yet (it wouldn't necessarily reproduce all the functionality from the built-in PV slicer yet).
I'm still working on this, but if anyone has any thoughts at this point, let me know! I plan to add a few more examples like this to the repo and then try and work on a branch of glue to see how easy it would be to make this happen.
(just to clarify, this is different from the project I was working on to develop a standalone PV extractor based on glue - what is here shows how a user would add a new tool to the button bar for image widgets, and connect this to a 'path' action that triggers a callback function - so the aim would be to replace https://github.com/glue-viz/glue/blob/master/glue/plugins/tools/pv_slicer.py with simpler code)

I don't really have time to pursue this, so I'm going to close it (since it was just an experiment)


Let's Encrypt HTTP challenge renewal fails with timeout

Describe the bug
Since about two months, certbot renewal of letsencypt certificates fails. This is persistent through several versions of NPM now and none of the exisiting issues such as fixing dns inside docker have solved the issue.
Nginx Proxy Manager Version
2.9.10
To Reproduce
Steps to reproduce the behavior:

Start docker
wait for renewal of soon expiring certificates
Check docker logs
Errors out with:

[11/2/2021] [7:07:02 PM] [Migrate  ] › ℹ  info      Current database version:<PHONE_NUMBER>3500
[11/2/2021] [7:07:02 PM] [Setup    ] › ℹ  info      Logrotate Timer initialized
[11/2/2021] [7:07:02 PM] [Setup    ] › ℹ  info      Logrotate completed.
[11/2/2021] [7:07:02 PM] [IP Ranges] › ℹ  info      Fetching IP Ranges from online services...
[11/2/2021] [7:07:02 PM] [IP Ranges] › ℹ  info      Fetching https://ip-ranges.amazonaws.com/ip-ranges.json
[11/2/2021] [7:07:06 PM] [IP Ranges] › ℹ  info      Fetching https://www.cloudflare.com/ips-v4
[11/2/2021] [7:07:11 PM] [IP Ranges] › ℹ  info      Fetching https://www.cloudflare.com/ips-v6
[11/2/2021] [7:07:15 PM] [SSL      ] › ℹ  info      Let's Encrypt Renewal Timer initialized
[11/2/2021] [7:07:15 PM] [SSL      ] › ℹ  info      Renewing SSL certs close to expiry...
[11/2/2021] [7:07:15 PM] [IP Ranges] › ℹ  info      IP Ranges Renewal Timer initialized
[11/2/2021] [7:07:15 PM] [Global   ] › ℹ  info      Backend PID 249 listening on port 3000 ...
`QueryBuilder#allowEager` method is deprecated. You should use `allowGraph` instead. `allowEager` method will be removed in 3.0
`QueryBuilder#eager` method is deprecated. You should use the `withGraphFetched` method instead. `eager` method will be removed in 3.0
QueryBuilder#omit is deprecated. This method will be removed in version 3.0
Model#$omit is deprected and will be removed in 3.0.
[11/2/2021] [7:09:05 PM] [SSL      ] › ✖  error     Error: Command failed: certbot renew --non-interactive --quiet --config "/etc/letsencrypt.ini" --preferred-challenges "dns,http" --disable-hook-validation
Failed to renew certificate npm-4 with error: Some challenges have failed.
Failed to renew certificate npm-5 with error: Some challenges have failed.
All renewals failed. The following certificates could not be renewed:
  /etc/letsencrypt/live/npm-4/fullchain.pem (failure)
  /etc/letsencrypt/live/npm-5/fullchain.pem (failure)
2 renew failure(s), 0 parse failure(s)

    at ChildProcess.exithandler (node:child_process:397:12)
    at ChildProcess.emit (node:events:390:28)
    at maybeClose (node:internal/child_process:1064:16)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:301:5)

Expected behavior
Certbot will automatically renew expiring certificates
Operating System
Linux 5.13.0-arm64 #1 SMP PREEMPT Debian 5.13.15-202109101456~buster (2021-09-10) aarch64 GNU/Linux
Additional context
I use the following ports, as my NPM is installed alongside Nextcloudpi, which by default occupies default HTTP(s) ports 80/443. Externally, ports 443 and 80 point towards 40443 and 8080, respectively. However, this was not a problem earlier (prior to ~August/September 2021).
version: '3'
services:
  app:
    image: 'jc21/nginx-proxy-manager:latest'
    restart: always
    ports:
      - '8080:80'
      - '8081:81'
      - '40443:443'
...

Unfortunately certbot does not output much information in the command line. Have a look at https://github.com/jc21/nginx-proxy-manager/issues/1271#user-content-certificate-error and tell us what the letsencrypt logs say

Thanks so much for the hint and sorry for missing this. Checking the letsencrypt logs revealed that the renewal fails due to the DNS challenge being invalid. That is reasonable, since I never set it up anyways. So I found https://letsencrypt.org/docs/challenge-types/ which leads me to believe, that for my use case simple HTTP-01 challenge is sufficient.
Maybe you could help me with the following question:
• is there any particular reason why NPM chooses DNS challenge by default?
• can I alter existing certificates to only use HTTP-01 challenges?
• and is there any immediate security implication to using HTTP-01 challenges only that I am missing?
Also, I think the following improvements could be added to NPM (probably deserving their own issues):
• add an info box on the toggle Use DNS Challenge to https://letsencrypt.org/docs/challenge-types/
• it is possible to generate a certificate on the fly when setting up a new proxy host. There is no mention of the DNS challenge. There should be a toggle as well, right?

Okay, that's weird. About your questions:

NPM does not use the DNS challenge by default, you have to check the box if you want to use it. Certbot then stores the information how it was requested and should reuse that information on renewals. This is stored in /etc/letsencrypt/renewal.
there is no way to alter the certificates later. But have a look at the certificates table, specifically at the Certificate Provider column. If it just says LetsEncrypt it was created as a HTTP challenge. If it is a DND challenge it should say  LetsEncrypt -<provider name>.
no there are no security implications. But if you don't want to expose your npm instance publicly, a http challenge won't work. And if you want to generate wildcard certificates, it also won't work.

I'm not really sure what the issue could be. Could you provide us with the relevant part of the letsencrypt log and the renewal config? Replace any sensitive information with placeholders of course.

Thanks so much for your help!
So I watched the logs while (successfully) creating a new certificate and deliberately not activating the DNS challenge. This triggered the following log in the docker app:
[11/3/2021] [8:05:18 PM] [SSL      ] › ℹ  info      Command: certbot certonly --non-interactive --config "/etc/letsencrypt.ini" --cert-name "npm-8" --agree-tos --authenticator webroot --email<EMAIL_ADDRESS>--preferred-challenges "dns,http" --domains "test.my-domain.tld"

Is it normal that certbot will include --preferred-challenges "dns,http" even though the DNS challenge was not ticked in NPM? I also tried to renew this newly created certificate and eveything worked fine, no LE-logs just a success log for NPM. Does that maybe mean that I am just better off by replacing all certificates?
Concerning previous certificates, please find the logs. I hope I did not blank relevant stuff. The part that tripped me up is that challenges contains {       "type": "dns-01",       "status": "pending",       "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/asdf",       "token": "asdf"     },. Also, in /etc/letsencrypt/renewal/npm-5.conf there is pref_challs = dns-01, http-01 while NPM's certificate table just says Let's Encrypt as Certificate Provider.
What do you mean by

But if you don't want to expose your npm instance publicly, [...]?

NPM, and the sites it proxies are exposed via ports 80 and 443 (external) and forwarded to 8080 and 40443 internally, but that should not matter, right? Port 81 (NPM interface) is not exposed externally.
Logs
2021-11-03 20:11:55,694:DEBUG:certbot._internal.plugins.selection:Requested authenticator webroot and installer None
2021-11-03 20:11:55,711:DEBUG:certbot._internal.plugins.selection:Single candidate plugin: * webroot
Description: Place files in webroot directory
Interfaces: Authenticator, Plugin
Entry point: webroot = certbot._internal.plugins.webroot:Authenticator
Initialized: <certbot._internal.plugins.webroot.Authenticator object at 0xffffbc593e48>
Prep: True
2021-11-03 20:11:55,712:DEBUG:certbot._internal.plugins.selection:Selected authenticator <certbot._internal.plugins.webroot.Authenticator object at 0xffffbc593e48> and installer None
2021-11-03 20:11:55,713:INFO:certbot._internal.plugins.selection:Plugins selected: Authenticator webroot, Installer None
2021-11-03 20:11:55,749:DEBUG:certbot._internal.main:Picked account: <Account(RegistrationResource(body=Registration(key=None, contact=(), agreement=None, status=None, terms_of_service_agreed=None, only_return_existing=None, external_account_binding=None), uri='https://acme-v02.api.letsencrypt.org/acme/acct/107220101', new_authzr_uri=None, terms_of_service=None), asdf, Meta(creation_dt=datetime.datetime(2020, 12, 23, 12, 6, 43, tzinfo=<UTC>), creation_host='72bcc8d98e39', register_to_eff=None))>
2021-11-03 20:11:55,752:DEBUG:acme.client:Sending GET request to https://acme-v02.api.letsencrypt.org/directory.
2021-11-03 20:11:55,760:DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org:443
2021-11-03 20:12:00,295:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "GET /directory HTTP/1.1" 200 658
2021-11-03 20:12:00,297:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Wed, 03 Nov 2021 20:12:00 GMT
Content-Type: application/json
Content-Length: 658
Connection: keep-alive
Cache-Control: public, max-age=0, no-cache
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "H0-MQorH-6Q": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417",
  "keyChange": "https://acme-v02.api.letsencrypt.org/acme/key-change",
  "meta": {
    "caaIdentities": [
      "letsencrypt.org"
    ],
    "termsOfService": "https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf",
    "website": "https://letsencrypt.org"
  },
  "newAccount": "https://acme-v02.api.letsencrypt.org/acme/new-acct",
  "newNonce": "https://acme-v02.api.letsencrypt.org/acme/new-nonce",
  "newOrder": "https://acme-v02.api.letsencrypt.org/acme/new-order",
  "revokeCert": "https://acme-v02.api.letsencrypt.org/acme/revoke-cert"
}
2021-11-03 20:12:00,301:DEBUG:certbot._internal.display.obj:Notifying user: Renewing an existing certificate for my-domain.tld
2021-11-03 20:12:00,414:DEBUG:certbot.crypto_util:Generating ECDSA key (2048 bits): /etc/letsencrypt/keys/1776_key-certbot.pem
2021-11-03 20:12:00,526:DEBUG:certbot.crypto_util:Creating CSR: /etc/letsencrypt/csr/1776_csr-certbot.pem
2021-11-03 20:12:00,528:DEBUG:acme.client:Requesting fresh nonce
2021-11-03 20:12:00,528:DEBUG:acme.client:Sending HEAD request to https://acme-v02.api.letsencrypt.org/acme/new-nonce.
2021-11-03 20:12:00,693:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "HEAD /acme/new-nonce HTTP/1.1" 200 0
2021-11-03 20:12:00,695:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Wed, 03 Nov 2021 20:12:00 GMT
Connection: keep-alive
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: asdf
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

2021-11-03 20:12:00,695:DEBUG:acme.client:Storing nonce: asdf
2021-11-03 20:12:00,697:DEBUG:acme.client:JWS payload:
b'{\n  "identifiers": [\n    {\n      "type": "dns",\n      "value": "my-domain.tld"\n    }\n  ]\n}'
2021-11-03 20:12:00,707:DEBUG:acme.client:Sending POST request to https://acme-v02.api.letsencrypt.org/acme/new-order:
{
  "protected": "asdf",
  "signature": "asdf",
  "payload": "asdf"
}
2021-11-03 20:12:01,036:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "POST /acme/new-order HTTP/1.1" 201 335
2021-11-03 20:12:01,038:DEBUG:acme.client:Received response:
HTTP 201
Server: nginx
Date: Wed, 03 Nov 2021 20:12:00 GMT
Content-Type: application/json
Content-Length: 335
Connection: keep-alive
Boulder-Requester: 107220101
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
Location: https://acme-v02.api.letsencrypt.org/acme/order/107220101/36894884650
Replay-Nonce: asdf
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "status": "pending",
  "expires": "2021-11-10T20:12:00Z",
  "identifiers": [
    {
      "type": "dns",
      "value": "my-domain.tld"
    }
  ],
  "authorizations": [
    "https://acme-v02.api.letsencrypt.org/acme/authz-v3/asdf"
  ],
  "finalize": "https://acme-v02.api.letsencrypt.org/acme/finalize/asdf"
}
2021-11-03 20:12:01,039:DEBUG:acme.client:Storing nonce: asdf
2021-11-03 20:12:01,040:DEBUG:acme.client:JWS payload:
b''
2021-11-03 20:12:01,048:DEBUG:acme.client:Sending POST request to https://acme-v02.api.letsencrypt.org/acme/authz-v3/45939634400:
{
  "protected": "asdf",
  "signature": "asdf",
  "payload": ""
}
2021-11-03 20:12:01,246:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "POST /acme/authz-v3/45939634400 HTTP/1.1" 200 793
2021-11-03 20:12:01,248:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Wed, 03 Nov 2021 20:12:01 GMT
Content-Type: application/json
Content-Length: 793
Connection: keep-alive
Boulder-Requester: 107220101
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: asdf
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "identifier": {
    "type": "dns",
    "value": "my-domain.tld"
  },
  "status": "pending",
  "expires": "2021-11-10T20:12:00Z",
  "challenges": [
    {
      "type": "http-01",
      "status": "pending",
      "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/asdf",
      "token": "asdf"
    },
    {
      "type": "dns-01",
      "status": "pending",
      "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/asdf",
      "token": "asdf"
    },
    {
      "type": "tls-alpn-01",
      "status": "pending",
      "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/asdf",
      "token": "asdf"
    }
  ]
}

This seems to keep going for a few tries until:
Server: nginx
Date: Wed, 03 Nov 2021 20:12:25 GMT
Content-Type: application/json
Content-Length: 1858
Connection: keep-alive
Boulder-Requester: 107220101
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: asdf
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "identifier": {
    "type": "dns",
    "value": "my-domain.tld"
  },
  "status": "invalid",
  "expires": "2021-11-10T20:12:00Z",
  "challenges": [
    {
      "type": "http-01",
      "status": "invalid",
      "error": {
        "type": "urn:ietf:params:acme:error:connection",
        "detail": "Fetching https://my-domain.tld/.well-known/acme-challenge/asdf: Timeout during connect (likely firewall problem)",
        "status": 400
      },
      "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/asdf",
      "token": "asdf",
      "validationRecord": [
        {
          "url": "http://my-domain.tld/.well-known/acme-challenge/asdf",
          "hostname": "my-domain.tld",
          "port": "80",
          "addressesResolved": [
            "some_ipv4_address",
            "some_ipv6_address"
          ],
          "addressUsed": "some_ipv6_address"
        },
        {
          "url": "http://my-domain.tld/.well-known/acme-challenge/asdf",
          "hostname": "my-domain.tld",
          "port": "80",
          "addressesResolved": [
            "some_ipv4_address",
            "some_ipv6_address"
          ],
          "addressUsed": "some_ipv4_address"
        },
        {
          "url": "https://my-domain.tld/.well-known/acme-challenge/asdf",
          "hostname": "my-domain.tld",
          "port": "443",
          "addressesResolved": [
            "some_ipv4_address",
            "some_ipv6_address"
          ],
          "addressUsed": "some_ipv6_address"
        }
      ],
      "validated": "2021-11-03T20:12:01Z"
    }
  ]
}
2021-11-03 20:12:25,204:DEBUG:acme.client:Storing nonce: asdf
2021-11-03 20:12:25,205:INFO:certbot._internal.auth_handler:Challenge failed for domain my-domain.tld
2021-11-03 20:12:25,206:INFO:certbot._internal.auth_handler:http-01 challenge for my-domain.tld
2021-11-03 20:12:25,207:DEBUG:certbot._internal.display.obj:Notifying user:
Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
  Domain: my-domain.tld
  Type:   connection
  Detail: Fetching https://my-domain.tld/.well-known/acme-challenge/asdf: Timeout during connect (likely firewall problem)

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

2021-11-03 20:12:25,208:DEBUG:certbot._internal.error_handler:Encountered exception:
Traceback (most recent call last):
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/auth_handler.py", line 90, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, best_effort)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/auth_handler.py", line 178, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.

2021-11-03 20:12:25,209:DEBUG:certbot._internal.error_handler:Calling registered functions
2021-11-03 20:12:25,209:INFO:certbot._internal.auth_handler:Cleaning up challenges
2021-11-03 20:12:25,210:DEBUG:certbot._internal.plugins.webroot:Removing /data/letsencrypt-acme-challenge/.well-known/acme-challenge/asdf
2021-11-03 20:12:25,211:DEBUG:certbot._internal.plugins.webroot:All challenges cleaned up
2021-11-03 20:12:25,212:ERROR:certbot._internal.renewal:Failed to renew certificate npm-4 with error: Some challenges have failed.
2021-11-03 20:12:25,216:DEBUG:certbot._internal.renewal:Traceback was:
Traceback (most recent call last):
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/renewal.py", line 475, in handle_renewal_request
    main.renew_cert(lineage_config, plugins, renewal_candidate)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/main.py", line 1386, in renew_cert
    renewed_lineage = _get_and_save_cert(le_client, config, lineage=lineage)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/main.py", line 122, in _get_and_save_cert
    renewal.renew_cert(config, domains, le_client, lineage)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/renewal.py", line 335, in renew_cert
    new_cert, new_chain, new_key, _ = le_client.obtain_certificate(domains, new_key)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/client.py", line 384, in obtain_certificate
    orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/client.py", line 434, in _get_order_and_authorizations
    authzr = self.auth_handler.handle_authorizations(orderr, self.config, best_effort)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/auth_handler.py", line 90, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, best_effort)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/auth_handler.py", line 178, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.

Ok, I why certificates would not be renewed: for the proxy hosts in question, I had the option "Force SSL" active. Once I deactivated this option in the SSL Tab of the respective hosts, I was able to renew all certificates. :heavy_check_mark:
Does leave this issue to be closed for "user error" (that could be totally on me. Sorry for spamming here...), or would that indicate a deeper problem worth keeping this issue around?

So that means enabling Force SSL caused the certificates to fail to renew?

Right now, I just can definitively state the inverse: disabling Force SSL made renewal possible. I tested it for 3 proxy hosts, all worked afterwards. I can test enabling Force SSL once more and check if that really is what caused the hickup

I am so confused and embarrassed... I tested whether enabling any of the SSL options Force SSL, HSTS Enabled and HSTS Subdomains would break the renewal process. I activated them one after the other and renewed the certificate every single time with success. At this point I don't know what has caused the incident, I am sorry. But maybe I'll write up the (admittedly super hacky) solution for future reference in a closing comment, if you think it could be helpful to the community @chaptergy. Just LMK, else feel free to close.

Hm, so now everything works no matter the state of any of the settings? You can't replicate the issue anymore? Then I'll go ahead and close this issue for now. But feel free to add anything that could be helpful in a comment.

Just for reference.  I have seen this behavior before if for some reason the existing certificate has lapsed and Force SSL is on then yes renewal will fail as it is forced to use a certificate that is expired and hence cant renew as the SSL in invalid. Maybe that is the situation you found yourself in.

thx for the hint - disabling Force SSL let me renew all SSL Certs

I was affected by the same bug. Disabling the SSL Force option allowed me to renew the cert as well.

Same here. After finding this issue today, i tried disabling "Force SSL" which indeed did the trick for 10 expired certificates. I kept recreateing certificates for over a year now without finding the issue.
I also tried one certificate which is still valid until May. With "Force SSL" enabled, renewal didnt work. As soon as i disabled the option, renewal worked.
Nevertheless, it cant be expired certificates because NPM should renew them before they expire.

I've had the same issue via the unRaid Docker container up until today, instead of disabling Force SSL I instead added a custom location rule (in the proxy host settings) with the following settings:
Location: /.well-known/acme-challenge
Scheme: http
Forward Hostname/IP: [YourNginxServerIP]/data/letsencrypt-acme-challenge (do not add your Nginx custom port here)
Forward Port: Your Nginx Port
This resolved the issue where the challenge files generated by the certification process could not be accessed by the remote host, my previous solution was to disable the proxy host temporarily, generate new certificate, and then re-enable it, only had to be done once every 3 months but it was still nonsensical.

I've ran into this issue twice now and just found this solution. The previous time I completely removed NPM and re-installed thinking there was something wrong with the installation. This saved me from having to repeat the process, thank you!


UnicodeDecodeError: 'utf8' codec can't decode byte 0x9c in position 1: invalid start byte

Expected Behavior

Bot running.
Actual Behavior

Error message, see traceback below.
Your FULL config.json (remove your username, password, gmapkey and any other private info)

http://pastebin.com/5X1ZKz2x
Output when issue occurred

http://pastebin.com/M3xHmRmS
Steps to Reproduce

Run the bot
Other Information
OS: CentOS

Branch: latest dev

Git Commit:<PHONE_NUMBER>f996f156fd00b1467d7f14afb1d3a4

Python Version: Python 2.7.5

Any other relevant files/configs (eg: path files)

Getting the exact same error output.

I hit into this and used VPN to resolve it. Looks like to me that my IP address was banned.

nope with different IP still gtting this erro.r

same probleme

same error. debian 8 box on google computing.
2016-08-25 14:04:07,100 [PokemonGoBot] [INFO] [login_started] Login procedure started. Traceback (most recent call last):   File "pokecli.py", line 705, in <module>     main()   File "pokecli.py", line 187, in main     report_summary(bot)   File "pokecli.py", line 221, in report_summary     metrics.capture_stats()   File "/opt/PokemonGo-Bot/pokemongo_bot/metrics.py", line 102, in capture_stats     response_dict = request.call()   File "/opt/PokemonGo-Bot/pokemongo_bot/api_wrapper.py", line 239, in call     result = self._call()   File "/opt/PokemonGo-Bot/pokemongo_bot/api_wrapper.py", line 163, in _call     accelerometer_axes=3   File "/opt/PokemonGo-Bot/local/lib/python2.7/site-packages/google/protobuf/internal/python_message.py", line 533, in init     setattr(self, field_name, field_value)   File "/opt/PokemonGo-Bot/local/lib/python2.7/site-packages/google/protobuf/internal/python_message.py", line 669, in field_setter     new_value = type_checker.CheckValue(new_value)   File "/opt/PokemonGo-Bot/local/lib/python2.7/site-packages/google/protobuf/internal/type_checkers.py", line 134, in CheckValue     raise ValueError('Value out of range: %d' % proposed_value) ValueError: Value out of range: -146 2016-08-25 14:04:09,729 [sentry.errors] [ERROR] Sentry responded with an error: 'utf8' codec can't decode byte 0x9c in position 1: invalid start byte (url: https://app.getsentry.com/api/90254/store/) Traceback (most recent call last):   File "/opt/PokemonGo-Bot/local/lib/python2.7/site-packages/raven/transport/threaded.py", line 174, in send_sync     super(ThreadedHTTPTransport, self).send(data, headers)   File "/opt/PokemonGo-Bot/local/lib/python2.7/site-packages/raven/transport/http.py", line 47, in send     ca_certs=self.ca_certs,   File "/opt/PokemonGo-Bot/local/lib/python2.7/site-packages/raven/utils/http.py", line 66, in urlopen     return opener.open(url, data, timeout)   File "/opt/PokemonGo-Bot/local/lib/python2.7/site-packages/future/backports/urllib/request.py", line 494, in open     response = self._open(req, data)   File "/opt/PokemonGo-Bot/local/lib/python2.7/site-packages/future/backports/urllib/request.py", line 512, in _open     '_open', req)   File "/opt/PokemonGo-Bot/local/lib/python2.7/site-packages/future/backports/urllib/request.py", line 466, in _call_chain     result = func(*args)   File "/opt/PokemonGo-Bot/local/lib/python2.7/site-packages/raven/utils/http.py", line 46, in https_open     return self.do_open(ValidHTTPSConnection, req)   File "/opt/PokemonGo-Bot/local/lib/python2.7/site-packages/future/backports/urllib/request.py", line 1284, in do_open     h.request(req.get_method(), req.selector, req.data, headers)   File "/usr/lib/python2.7/httplib.py", line 1001, in request     self._send_request(method, url, body, headers)   File "/usr/lib/python2.7/httplib.py", line 1035, in _send_request     self.endheaders(body)   File "/usr/lib/python2.7/httplib.py", line 997, in endheaders     self._send_output(message_body)   File "/usr/lib/python2.7/httplib.py", line 848, in _send_output     msg += message_body   File "/opt/PokemonGo-Bot/lib/python2.7/encodings/utf_8.py", line 16, in decode     return codecs.utf_8_decode(input, errors, True) UnicodeDecodeError: 'utf8' codec can't decode byte 0x9c in position 1: invalid start byte 2016-08-25 14:04:09,731 [sentry.errors.uncaught] [ERROR] [u'ValueError: Value out of range: -146', u'  File "pokecli.py", line 705, in <module>', u'  File "pokecli.py", line 187, in main', u'  File "pokecli.py", line 221, in report_summary', u'  File "pokemongo_bot/metrics.py", line 102, in capture_stats', u'  File "pokemongo_bot/api_wrapper.py", line 239, in call', u'  File "pokemongo_bot/api_wrapper.py", line 163, in _call', u'  File "/opt/PokemonGo-Bot/local/lib/python2.7/site-packages/google/protobuf/internal/python_message.py", line 533, in init', u'  File "/opt/PokemonGo-Bot/local/lib/python2.7/site-packages/google/protobuf/internal/python_message.py", line 669, in field_setter', u'  File "/opt/PokemonGo-Bot/local/lib/python2.7/site-packages/google/protobuf/internal/type_checkers.py", line 134, in CheckValue'] Thu Aug 25 14:04:09 UTC 2016 Pokebot  Stopped.

Same error on here. I fired up my VPN and changed IP's to no avail.

Changing "health_record": true, to "health_record": false, took away some errors.
Now it is just:
Traceback (most recent call last):   File "pokecli.py", line 705, in <module>     main()   File "pokecli.py", line 104, in main     bot.start()   File "/PokemonGo-Bot/pokemongo_bot/__init__.py", line 109, in start     init_inventory(self)   File "/PokemonGo-Bot/pokemongo_bot/inventory.py", line 1219, in init_inventory     _inventory = Inventory(bot)   File "/PokemonGo-Bot/pokemongo_bot/inventory.py", line 1086, in __init__     self.refresh()   File "/PokemonGo-Bot/pokemongo_bot/inventory.py", line 1096, in refresh     self.update_web_inventory()   File "/PokemonGo-Bot/pokemongo_bot/inventory.py", line 1103, in update_web_inventory     json_inventory = json.load(infile)   File "/usr/lib64/python2.7/json/__init__.py", line 290, in load     **kw)   File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads     return _default_decoder.decode(s)   File "/usr/lib64/python2.7/json/decoder.py", line 365, in decode     obj, end = self.raw_decode(s, idx=_w(s, 0).end())   File "/usr/lib64/python2.7/json/decoder.py", line 381, in raw_decode     obj, end = self.scan_once(s, idx) ValueError: Expecting object: line 1 column 6006 (char 6005)
Changing
{
         "type": "UpdateWebInventory",
         "config": {
            "enabled": true
         }
      },

To
{           "type": "UpdateWebInventory",          "config": {             "enabled": false          }       },
Doesnt change anything.

@davidspeijer Do you have this file web/inventory-botname.json? If so - whats inside? Can you try to clear it?

@sohje I don't have it.... and yes, same error as OP

@r3pek could you please make file web/inventory-botname_here.json and try again?

Clearing the file result in:

Traceback (most recent call last):
File "pokecli.py", line 705, in
main()
File "pokecli.py", line 104, in main
bot.start()
File "/PokemonGo-Bot/pokemongo_bot/init.py", line 109, in start
init_inventory(self)
File "/PokemonGo-Bot/pokemongo_bot/inventory.py", line 1219, in init_inventory
_inventory = Inventory(bot)
File "/PokemonGo-Bot/pokemongo_bot/inventory.py", line 1086, in init
self.refresh()
File "/PokemonGo-Bot/pokemongo_bot/inventory.py", line 1096, in refresh
self.update_web_inventory()
File "/PokemonGo-Bot/pokemongo_bot/inventory.py", line 1103, in update_web_inventory
json_inventory = json.load(infile)
File "/usr/lib64/python2.7/json/init.py", line 290, in load
**kw)
File "/usr/lib64/python2.7/json/init.py", line 338, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python2.7/json/decoder.py", line 365, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python2.7/json/decoder.py", line 383, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

Nop... same problem. Either with empty file or with empty JSON
2016-08-25 15:54:04,825 [PokemonGoBot] [INFO]  Traceback (most recent call last):   File "pokecli.py", line 705, in <module>     main()   File "pokecli.py", line 187, in main     report_summary(bot)   File "pokecli.py", line 221, in report_summary     metrics.capture_stats()   File "/root/PokemonGo-Bot/pokemongo_bot/metrics.py", line 102, in capture_stats     response_dict = request.call()   File "/root/PokemonGo-Bot/pokemongo_bot/api_wrapper.py", line 239, in call     result = self._call()   File "/root/PokemonGo-Bot/pokemongo_bot/api_wrapper.py", line 188, in _call     return PGoApiRequest.call(self, signature) TypeError: call() takes exactly 1 argument (2 given) 2016-08-25 15:54:04,905 [sentry.errors] [ERROR] Sentry responded with an error: 'utf8' codec can't decode byte 0x9c in position 1: invalid start byte (url: https://app.getsentry.com/api/90254/store/) Traceback (most recent call last):   File "/usr/lib/python2.7/site-packages/raven/transport/threaded.py", line 174, in send_sync     super(ThreadedHTTPTransport, self).send(data, headers)   File "/usr/lib/python2.7/site-packages/raven/transport/http.py", line 47, in send     ca_certs=self.ca_certs,   File "/usr/lib/python2.7/site-packages/raven/utils/http.py", line 66, in urlopen     return opener.open(url, data, timeout)   File "/usr/lib/python2.7/site-packages/future/backports/urllib/request.py", line 494, in open     response = self._open(req, data)   File "/usr/lib/python2.7/site-packages/future/backports/urllib/request.py", line 512, in _open     '_open', req)   File "/usr/lib/python2.7/site-packages/future/backports/urllib/request.py", line 466, in _call_chain     result = func(*args)   File "/usr/lib/python2.7/site-packages/raven/utils/http.py", line 46, in https_open     return self.do_open(ValidHTTPSConnection, req)   File "/usr/lib/python2.7/site-packages/future/backports/urllib/request.py", line 1284, in do_open     h.request(req.get_method(), req.selector, req.data, headers)   File "/usr/lib/python2.7/httplib.py", line 1057, in request     self._send_request(method, url, body, headers)   File "/usr/lib/python2.7/httplib.py", line 1097, in _send_request     self.endheaders(body)   File "/usr/lib/python2.7/httplib.py", line 1053, in endheaders     self._send_output(message_body)   File "/usr/lib/python2.7/httplib.py", line 895, in _send_output     msg += message_body   File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode     return codecs.utf_8_decode(input, errors, True) UnicodeDecodeError: 'utf8' codec can't decode byte 0x9c in position 1: invalid start byte 2016-08-25 15:54:04,913 [sentry.errors.uncaught] [ERROR] [u'TypeError: call() takes exactly 1 argument (2 given)', u'  File "pokecli.py", line 705, in <module>', u'  File "pokecli.py", line 187, in main', u'  File "pokecli.py", line 221, in report_summary', u'  File "pokemongo_bot/metrics.py", line 102, in capture_stats', u'  File "pokemongo_bot/api_wrapper.py", line 239, in call', u'  File "pokemongo_bot/api_wrapper.py", line 188, in _call']

Removing resulting in

Traceback (most recent call last):
File "pokecli.py", line 705, in
main()
File "pokecli.py", line 104, in main
bot.start()
File "/PokemonGo-Bot/pokemongo_bot/init.py", line 109, in start
init_inventory(self)
File "/PokemonGo-Bot/pokemongo_bot/inventory.py", line 1219, in init_inventory
_inventory = Inventory(bot)
File "/PokemonGo-Bot/pokemongo_bot/inventory.py", line 1086, in init
self.refresh()
File "/PokemonGo-Bot/pokemongo_bot/inventory.py", line 1096, in refresh
self.update_web_inventory()
File "/PokemonGo-Bot/pokemongo_bot/inventory.py", line 1102, in update_web_inventory
with open(web_inventory, "r") as infile:
IOError: [Errno 2] No such file or directory: u'/PokemonGo-Bot/pokemongo_bot/../web/inventory-***.json'

@davidspeijer @r3pek  Could you put this into file?
[]

or
{}

@sohje same problem...

@sohje {} solved it for me... :)

@davidspeijer good ;]

@davidspeijer changed anything else besides adding {} to the file?

No, my original config is now working without any problems :).

thx.... I open to ideas then ;) this is a fresh install...

OK, mine is fixed too... pgoapi was too old ;)

just an update, occasionally i still get this error.... so, it's not  completely solved

I've put in a PR to fix this (create a "default" inventory file if it doesn't already exist). Hopefully that will resolve it.
@r3pek This specific error should only occur if the inventory file doesn't exist. When you say it "occasionally" still happens, is it exactly the same or something different? I'd like to know so I can track/fix any other bugs.

This is what i have at start now (can't start it again, even with "empty" inventory)
http://pastebin.com/yLpVuvA7

@r3pek create inventory file just with [] at the top (nothing else) and restart. Does that work?

@Gobberwart no. same error. with {} or []

Same error as Issue #4380. Just at another place

@r3pek looks like your error is different and not related to inventory json file.

yeah.... same error message though :(

@r3pek I assume you've tried updating requirements? This looks like pgoapi error which might be solved by updating.
Try: pip install -r requirements.txt --upgrade

Nothing to upgrade. Same problem.
On Aug 26, 2016 02:27, "Stuart Travers"<EMAIL_ADDRESS>wrote:

@r3pek https://github.com/r3pek I assume you've tried updating
requirements? This looks like pgoapi error which might be solved by
updating.
Try: pip install -r requirements.txt --upgrade
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/PokemonGoF/PokemonGo-Bot/issues/4718#issuecomment-242595238,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AC6MPNTjUgkrWePCss2R5XKiYGt0QziVks5qjkDlgaJpZM4Js3Rf
.

Please paste your most recent output/error

It's the one on the pastebin. Nothing changed.
On Aug 26, 2016 02:39, "Stuart Travers"<EMAIL_ADDRESS>wrote:

Please paste your most recent output/error
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/PokemonGoF/PokemonGo-Bot/issues/4718#issuecomment-242598131,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AC6MPBtGxCo1GWiKNMZHOIdCcHAa8xn1ks5qjkPngaJpZM4Js3Rf
.

Hmmm have you got the latest build?

Yes... Running dev branch.
On Aug 26, 2016 02:52, "Stuart Travers"<EMAIL_ADDRESS>wrote:

Hmmm have you got the latest build?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/PokemonGoF/PokemonGo-Bot/issues/4718#issuecomment-242600836,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AC6MPEdKPA9FXF3oaSDfbDfmh1GuE80Hks5qjkbDgaJpZM4Js3Rf
.

Latest commit 3622c032257887e5898778635c7382017879dafe fixes the problem for me.

commit 09403f5 also works

Since the original poster's issue is resolved, I am closing.

I have ths issues since last update this morning.
When calling for  Most Perfect Pokemon, when starting Bot.
[2016-08-30 10:46:20] [       cli] [INFO] Most Perfect Pokemon:  Traceback (most recent call last):   File "pokecli.py", line 768, in <module>     main()   File "pokecli.py", line 119, in main     bot = start_bot(bot,config)   File "pokecli.py", line 86, in start_bot     bot.start()   File "/home/guillaume/Applications/PokemonGo-Bot/pokemongo_bot/__init__.py", line 127, in start     init_inventory(self)   File "/home/guillaume/Applications/PokemonGo-Bot/pokemongo_bot/inventory.py", line 1273, in init_inventory     _inventory = Inventory(bot)   File "/home/guillaume/Applications/PokemonGo-Bot/pokemongo_bot/inventory.py", line 1111, in __init__     self.refresh()   File "/home/guillaume/Applications/PokemonGo-Bot/pokemongo_bot/inventory.py", line 1121, in refresh     self.update_web_inventory()   File "/home/guillaume/Applications/PokemonGo-Bot/pokemongo_bot/inventory.py", line 1148, in update_web_inventory     self.bot.logger.info('[x] Error while opening inventory file for read: %s' % e, 'red') NameError: global name 'e' is not defined [2016-08-30 10:46:20] [sentry.errors] [ERROR] Sentry responded with an error: 'utf8' codec can't decode byte 0x9c in position 1: invalid start byte (url: https://app.getsentry.com/api/90254/store/) Traceback (most recent call last):   File "/home/guillaume/.local/lib/python2.7/site-packages/raven/transport/threaded.py", line 174, in send_sync     super(ThreadedHTTPTransport, self).send(data, headers)   File "/home/guillaume/.local/lib/python2.7/site-packages/raven/transport/http.py", line 47, in send     ca_certs=self.ca_certs,   File "/home/guillaume/.local/lib/python2.7/site-packages/raven/utils/http.py", line 66, in urlopen     return opener.open(url, data, timeout)   File "/home/guillaume/.local/lib/python2.7/site-packages/future/backports/urllib/request.py", line 494, in open     response = self._open(req, data)   File "/home/guillaume/.local/lib/python2.7/site-packages/future/backports/urllib/request.py", line 512, in _open     '_open', req)   File "/home/guillaume/.local/lib/python2.7/site-packages/future/backports/urllib/request.py", line 466, in _call_chain     result = func(*args)   File "/home/guillaume/.local/lib/python2.7/site-packages/raven/utils/http.py", line 46, in https_open     return self.do_open(ValidHTTPSConnection, req)   File "/home/guillaume/.local/lib/python2.7/site-packages/future/backports/urllib/request.py", line 1284, in do_open     h.request(req.get_method(), req.selector, req.data, headers)   File "/usr/lib/python2.7/httplib.py", line 1057, in request     self._send_request(method, url, body, headers)   File "/usr/lib/python2.7/httplib.py", line 1097, in _send_request     self.endheaders(body)   File "/usr/lib/python2.7/httplib.py", line 1053, in endheaders     self._send_output(message_body)   File "/usr/lib/python2.7/httplib.py", line 895, in _send_output     msg += message_body   File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode     return codecs.utf_8_decode(input, errors, True) UnicodeDecodeError: 'utf8' codec can't decode byte 0x9c in position 1: invalid start byte [2016-08-30 10:46:20] [sentry.errors.uncaught] [ERROR] [u"NameError: global name 'e' is not defined", u'  File "pokecli.py", line 768, in <module>', u'  File "pokecli.py", line 119, in main', u'  File "pokecli.py", line 86, in start_bot', u'  File "pokemongo_bot/__init__.py", line 127, in start', u'  File "pokemongo_bot/inventory.py", line 1273, in init_inventory', u'  File "pokemongo_bot/inventory.py", line 1111, in __init__', u'  File "pokemongo_bot/inventory.py", line 1121, in refresh', u'  File "pokemongo_bot/inventory.py", line 1148, in update_web_inventory'] mardi 30 août 2016, 10:46:20 (UTC+0200) Pokebot  Stopped.

@castorfou Try PR #4924

I have a similiar error  "UnicodeDecodeError: 'utf8' codec can't decode byte 0x81 in position 38: invalid start byte",  happens after 400, message bad request syntax

<IP_ADDRESS> - - [06/Sep/2016 18:23:12] code 400, message Bad request syntax ("\x81\xfe\x00\x941\xb1\xb4\x1d\x05\x83\xef?S\xde\xc0'S\xc3\xdb|U\xd2\xd5nE\x93\x98=J\x93\xd5~R\xde\xc1sE\x93\x8e=\x13\xc1\xdbvT\xc2\xd7|_\xee\xc7z\x13\x9d\x94?U\xd0\xc0|\x13\x8b\x94f\x13\xdc\xc7z\x13\x8b\x94?u\xd8\xc7~P\xc3\xd0xU\x91\x87e\x11\xe1\xdbiX\xde\xda3\x13\x9d\x94?X\xc5\xd1p\x13\x8b\x94?a\xde\xc0t^\xdf\x961\x11\x93\xd5p^\xc4\xdai\x13\x8b\x94?\x02\x93\xc91\x11\x93\xd1kT\xdf\xc0?\x0b\x91\x96tE\xd4\xd9BU\xd8\xc7~P\xc3\xd0xU\x93\xc9@\x81\xfe\x013\x90\xc7\xf8\xa0\xa4\xf5\xa3\x82\xf2\xa8\x8c\x9a\xf2\xb5\x97\xc1\xf4\xa4\x99\xd3\xe4\xe5\xd4\x80\xeb\xe5\x99\xc3\xf3\xa8\x8d\xce\xe4\xe5\xc2\x80\xb2\xb7\x97\xcb\xf5\xb4\x9b\xc1\xfe\x98\x8b\xc7\xb2\xeb\xd8\x82\xf4\xa6\x8c\xc1\xb2\xfd\xd8\xdb\xb2\xb7\x97\xcb\xf5\xaa\x97\xce\xcf\xae\x9c\x82\xaa\xe7\xc9\x93\xbc\xe7\xda\xc5\xe8\xb7\x91\xd2\xf1\xb3\x91\xcf\xfe\x98\x8c\xc9\xfd\xa2\x8b\xd4\xf1\xaa\x88\xff\xfd\xb4\xda\x9a\xb0\xf6\xcc\x97\xa3\xf6\xcd\x97\xa5\xf3\xca\x98\xa1\xf5\xd4\x80\xb2\xab\x97\xce\xf7\xae\x8c\xd5\xf4\xa2\xda\x9a\xb0\xf6\xc8\x93\xbe\xff\xcb\x93\xa1\xf0\xcb\x99\xa6\xf0\xc1\x92\xa0\xf5\xca\x8c\xb0\xe5\x88\xcf\xfb\xa2\x95\xcf\xfe\x98\x96\xc1\xfd\xa2\xda\x9a\xb0\xe5\xaf\xc5\xf5\xa3\x94\xc5\xb2\xeb\xd8\x82\xfc\xa6\x8c\xc9\xe4\xb2\x9c\xc5\xb2\xfd\xd8\x91\xbe\xf4\xc8\x96\xa4\xf3\xcd\x97\xa2\xf3\xc1\x95\xa8\xf2\xc9\x93\xa5\xeb\xd8\x82\xe3\xb7\x99\xd7\xfe\x98\x88\xcf\xf9\xa9\x8c\xff\xf9\xa3\xda\x9a\xb0\xe5\xcb\x91\xf4\xa6\xc9\x99\xa8\xa3\xce\x91\xf6\xe5\xd4\x80\xb2\xa2\x96\xc3\xff\xb2\x96\xd4\xf5\xb5\xa7\xc9\xf4\xe5\xc2\x80\xa9\xf4\xc9\x92\xa3\xf2\xca\x92\xa2\xf2\xc0\x94\xa8\xf0\xcf\x98\xa2\xf6\x85\x8c\xb0\xe5\x9d\xd6\xf5\xa9\x8c\x82\xaa\xe7\xda\xc3\xf1\xb3\x9b\xc8\xf1\xa5\x94\xc5\xcf\xb7\x97\xcb\xf5\xaa\x97\xce\xb2\xba\xa5\x81\xfe\x013X\x99\x9d\xbdl\xab\xc6\x9f:\xf6\xe9\x87:\xeb\xf2\xdc<\xfa\xfc\xce,\xbb\xb1\x9d#\xbb\xfc\xde;\xf6\xe8\xd3,\xbb\xa7\x9dz\xe9\xf2\xd6=\xea\xfe\xdc6\xc6\xee\xdaz\xb5\xbd\x9f<\xf8\xe9\xdcz\xa3\xbd\xc6z\xe9\xf2\xd6=\xf4\xf2\xd3\x07\xf0\xf9\x9fb\xb9\xa9\x8bt\xb9\xbf\xd8 \xe9\xf4\xcf9\xed\xf4\xd26\xc6\xe9\xd45\xfc\xee\xc99\xf4\xed\xe25\xea\xbf\x87x\xa8\xa9\x8ak\xa8\xa8\x85h\xae\xa9\x8eh\xa1\xb1\x9dz\xf5\xf2\xd3?\xf0\xe9\xc8<\xfc\xbf\x87x\xa8\xad\x8ev\xa1\xae\x8eh\xa1\xae\x8ea\xab\xa8\x88h\xae\xa8\x91x\xbb\xed\xd23\xfc\xf0\xd26\xc6\xf3\xdc5\xfc\xbf\x87x\xbb\xcd\xdc*\xf8\xee\x9ft\xb9\xbf\xd19\xed\xf4\xc9-\xfd\xf8\x9fb\xb9\xac\x93k\xa9\xab\x85\xac\xac\x8bh\xab\xa8\x8co\xac\xaf\x91x\xbb\xee\xcd9\xee\xf3\xe2(\xf6\xf4\xd3,\xc6\xf4\xd9z\xa3\xbd\x9fk\xa8\xf9\xdci\xa0\xa5\xd9m\xff\xac\x9ft\xb9\xbf\xd86\xfa\xf2\xc86\xed\xf8\xcf\x07\xf0\xf9\x9fb\xb9\xac\x8eh\xa1\xa8\x8am\xa8\xac\x85\xa9\xae\x88j\xaa\xad\x89a\xaa\xe0\x91x\xbb\xf8\xcb=\xf7\xe9\x9fb\xb9\xbf\xde9\xed\xfe\xd59\xfb\xf1\xd8\x07\xe9\xf2\xd6=\xf4\xf2\xd3z\xe4\xc0\x81\xfe\x01S\x9alKH\xae^\x10j\xf8\x03?r\xf8\x1e$)\xfe\x0f*;\xeeNgh\xe1N*+\xf9\x03>&\xeeNqh\xb8\x1c$#\xff\x1f()\xf438/\xb8@kj\xfe\r?)\xb8Vk3\xb8")
Traceback (most recent call last):
File "/home/prusswan/Projects/PokemonGo-Bot/local/lib/python2.7/site-packages/eventlet/greenpool.py", line 82, in _spawn_n_impl
func(*args, **kwargs)
File "/home/prusswan/Projects/PokemonGo-Bot/local/lib/python2.7/site-packages/eventlet/wsgi.py", line 719, in process_request
proto.init(sock, address, self)
File "/usr/lib/python2.7/SocketServer.py", line 652, in init
self.handle()
File "/usr/lib/python2.7/BaseHTTPServer.py", line 342, in handle
self.handle_one_request()
File "/home/prusswan/Projects/PokemonGo-Bot/local/lib/python2.7/site-packages/eventlet/wsgi.py", line 351, in handle_one_request
if not self.parse_request():
File "/usr/lib/python2.7/BaseHTTPServer.py", line 286, in parse_request
self.send_error(400, "Bad request syntax (%r)" % requestline)
File "/usr/lib/python2.7/BaseHTTPServer.py", line 365, in send_error
self.send_response(code, message)
File "/usr/lib/python2.7/BaseHTTPServer.py", line 396, in send_response
self.log_request(code)
File "/usr/lib/python2.7/BaseHTTPServer.py", line 433, in log_request
self.requestline, str(code), str(size))
File "/usr/lib/python2.7/BaseHTTPServer.py", line 469, in log_message
format%args))
File "/home/prusswan/Projects/PokemonGo-Bot/lib/python2.7/codecs.py", line 369, in write
data, consumed = self.encode(object, self.errors)
File "/home/prusswan/Projects/PokemonGo-Bot/lib/python2.7/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x81 in position 38: invalid start byte

Same here.. Output in case it helps:
<IP_ADDRESS> - - [06/Sep/2016 13:59:42] code 400, message Bad request syntax ('\x81\xfe\x00\xd4\x1c\x7fw\xd9(M,\xfb~\x10\x03\xe3~\r\x18\xb8x\x1c\x16\xaah][\xf9g]\x16\xba\x7f\x10\x02\xb7h]M\xf9>\x14\x16\xb7}\x12\x12\xef%IN\xfb0_U\xbd}\x0b\x16\xfb&_\x0c\xfbq\x0c\x10\xfb&_U\x9c{\x18\x04\xf9u\x11\x14\xac~\x1e\x03\xb0r\x18M\xf9GOY\xef+PB\xf7,_\x1c\xb4A__\x9c{\x18\x04\xf9p\x1a\x11\xad&_A\xf5<6\x19\xbai\x1d\x16\xadu\x11\x10\xe3<N^\xfb0_U\xbc{\x18\x04\xfb&_U\xe92I@\xf6)QG\xf9w\x12U\xf5<]\x12\xbe{\x0c(\xb5y\x19\x03\xfb&_A\xf5<]\x12\xbe{\x0c(\xb0r\x1cU\xe3<N')  Traceback (most recent call last):    File "/usr/lib/python2.7/site-packages/eventlet/greenpool.py", line 82, in _spawn_n_impl      func(*args, **kwargs)    File "/usr/lib/python2.7/site-packages/eventlet/wsgi.py", line 719, in process_request      proto.__init__(sock, address, self)    File "/usr/lib/python2.7/SocketServer.py", line 652, in __init__      self.handle()    File "/usr/lib/python2.7/BaseHTTPServer.py", line 342, in handle      self.handle_one_request()    File "/usr/lib/python2.7/site-packages/eventlet/wsgi.py", line 351, in handle_one_request      if not self.parse_request():    File "/usr/lib/python2.7/BaseHTTPServer.py", line 286, in parse_request      self.send_error(400, "Bad request syntax (%r)" % requestline)    File "/usr/lib/python2.7/BaseHTTPServer.py", line 365, in send_error      self.send_response(code, message)    File "/usr/lib/python2.7/BaseHTTPServer.py", line 396, in send_response      self.log_request(code)    File "/usr/lib/python2.7/BaseHTTPServer.py", line 433, in log_request      self.requestline, str(code), str(size))    File "/usr/lib/python2.7/BaseHTTPServer.py", line 469, in log_message      format%args))    File "/usr/lib/python2.7/codecs.py", line 369, in write      data, consumed = self.encode(object, self.errors)    File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode      return codecs.utf_8_decode(input, errors, True)  UnicodeDecodeError: 'utf8' codec can't decode byte 0x81 in position 38: invalid start byte  [Errno 104] Connection reset by peer


Convert the Job controller to a sync controller

/assign @jianhuiz @irfanurrehman

/ok-to-test

/retest

/retest

@font how did u run the e2e for this change? It seems to fail always for jobs. Can you please have a look.

@irfanurrehman I was trying to use the prow job to perform e2e testing as there were some issues running it locally. Please hold off reviewing until I get it straightened out.

@irfanurrehman I see that in the code now, thanks! This should be documented as one is expecting jobs to be fully enabled just like the rest of the APIs. https://kubernetes.io/docs/tasks/administer-federation/job/ does not mention this.

@font, thanks for pointing this out. I have raised #198 for this.


Add instructions on how to build extensions using xk6 docker image

docker run --rm -e GOOS=darwin -u "$(id -u):$(id -g)" -v "${PWD}:/xk6" \
  grafana/xk6 build \
  --with xk6-internal=.`

Resolved in https://github.com/grafana/quickpizza/pull/4


Helm "enable metrics" boolean

Expected Behavior
A number of common helm charts have a value you can set to enable metrics reporting.  Since calico has a metrics interface, having a clean way to enable it via helm would be nice.
Current Behavior
You must manually patch the instance to enable metrics
Possible Solution
Add a boolean to values.yml and the relevant templates to enable the metrics service on request.
Steps to Reproduce (for bugs)

Install via helm
look for way to enable metrics as a parameter

Context

A consistent way to enable metrics across a wide array of helm charts makes things easier.
Your Environment

Calico version: 3.24.2
Orchestrator version (e.g. kubernetes, mesos, rkt): 1.25.2
Operating System and version: CentOS Stream9
Link to your project (optional):

Neat, I didn't see those in there.  Can it be added to the documentation?

Yeah, the documentation isn't very clear at the moment. We need to do better.
The installation section contains all of the fields specified in the InstallationSpec here: https://projectcalico.docs.tigera.io/reference/installation/api#operator.tigera.io/v1.InstallationSpec
Maybe there's a way we can auto-generate the values.yaml doc to include all of those fields, but for now that link is effectively the documentation.

@jcpunk out of curiosity, which pages of the documentation do you normally look at for this? We have the fields in https://projectcalico.docs.tigera.io/reference/installation/api but it seems that it's not a great UX. Any feedback on how we can better arrange this information is welcome.

OK, sounds like the right path forward is to beef up the documentation in the README for now. If there are any other suggestions you have on how to make this easier to read going forward, please let us know!

OK, sounds like the right path forward is to beef up the documentation in the README for now. If there are any other suggestions you have on how to make this easier to read going forward, please let us know!

@mgleung   @caseydavenport  Automatically generating the README.md document for the Chart could make the project easier to deploy. If there are enhancement plans, I think I could do this work and the following are the planned steps.

Select a tool for automatically generating the README.md document.

https://github.com/norwoodj/helm-docs
https://github.com/bitnami-labs/readme-generator-for-helm

Add comments to the values.yaml.
Add a step to the Makefile, for example make helm-chart-docs.


Error when reaching the end of Epoch1

File "train.py", line 140, in
ap ,mAP = voc_eval(eval_ds, model, ds.num_classes)
File "C:\Users\anind\Documents\Dissertation Codes\PhaseII_Novelty\tf2-faster-rcnn\eval.py", line 205, in voc_eval
precision, recall = voc_eval_pre_rec(dataset, model, num_classes)
File "C:\Users\anind\Documents\Dissertation Codes\PhaseII_Novelty\tf2-faster-rcnn\eval.py", line 35, in voc_eval_pre_rec
labels, bboxes, cls_prob = model(img_input)
ValueError: too many values to unpack (expected 3)

@mlnewbie1996 How do you solve this?


loss calculate

Sorry to bother you again. I am also confused with the loss function.
Can you make an brief explanation about how it calculate?
I see that you set the criterion in the file train.lua and pass the parameter to ResumableDPOptimEngine.
local criterion = nn.CrossEntropyCriterion(critweights, false):cuda() and I am lost in the file==.
After some steps,  I found that
runTest(state, set, meter)
local loss = meter:value() / math.log(2)

I guess that we calculate loss use the cross-entropy cost function of every batch. And add them together.
And do something others.

You're on the right track. The criterion computes the cross-entropy loss on forward(), and the loss of each worker is summed up in the engine when training (see https://github.com/facebookresearch/fairseq/blob/master/fairseq/torchnet/ResumableDPOptimEngine.lua#L375-L378 for the relevant lines). Please note that we compute the sum of the loss over all tokens, and thanks to the criterion weights we ignore padding tokens in the computation. The updateMeters hook is called after each sample and updates the meters with the latest loss value (https://github.com/facebookresearch/fairseq/blob/master/fairseq/torchnet/hooks.lua#L143). We then log the average loss over all non-padding tokens in the current epoch in https://github.com/facebookresearch/fairseq/blob/master/train.lua#L356.

Thank you very much. Got them.


Remove dependency on Hurley

The hurley gem is no longer being maintained.

This was already done in the v0.10 branch. If you'd like, I can finish up that work and merge it in.

That would be a huge help!


Fixed #26007 - get_template_names returns complete stack of names.

SingleObjectTemplateResponseMixin.get_template_names() now returns a list of
names matching the documentation, rather than only template_name if that
was specified. Existing code implies that the name returned from
template_name_field should have been first, so doc was updated to change
the ordering. New test case added. Sphinx docs updated.

Buildbot, add to whitelist.

Is this backwards-compatible?

One backward incompatible change is if the developer sets template_name AND template_name_field AND the object has a value in that field, this method will now return the latter rather than the former. It seems clear, though, that the intent of the code was always to return the result of template_name_field.
A related issue: A test of hasattr(self,'object') was added in two places to cover the case that a developer used this mixin, set a template_name, but did not set self.object. Without that addition to the patch, this patch would start raising ImproperlyConfigured.

I don't see what implies that template_name_field should be first? Even if the code implies some other behavior, it seems to me that the the documentation and docstring should take precedence.

@timgraham, this one's been a while! Give me a bit to wade through the code again to see if my logic still makes sense :)

Closing due to inactivity.


Dump LruItem size

coverage: 99.263% (-0.05%) from 99.315%
when pulling 37829802f5cc049148fe9ae78c67d6313d716fd8 on users/alexpeck/lrualign
into 1621b211c00d83ca1b0011173bf33deb912efdc9 on main.


Url acl prefix fix

Fixes #2038

@khellang can you stop breaking Team City...

@phillip-haydon Totally unrelated. Rebuilt and all is :ok:

Pfft I know you're trying to sabotage the Nancy project!


Reproducing RoBERTa*_ORA_P_D(last step) results

Hi,
I am attempting to reproduce the numbers for the dev set for RoBERTa*_ORA_P_D(last step).
I am using downloaded checkpoints to minimise variability:

generated decompositions with transformer_qa_large.tar.gz -> transformer_qa_ORA-P_dev_no_placeholders.json & transformer_qa_ORA-P_train_no_placeholders.json
8_STAR_ORA-P-D-last-step.tar.gz

My output when running this command
$ python3 run_scripts/evaluate.py --model pre-trained_models/8_STAR_ORA-P-D-last-step/8_STAR_ORA-P-D-last-step.tar.gz --data data/strategyqa/dev.json -g "1"
[Lots of output, not included]
Traceback (most recent call last):
  File "/nfs/strategyqa/run_scripts/run.py", line 45, in run
    allennlp_main()
  File "/usr/local/lib/python3.6/dist-packages/allennlp/commands/__init__.py", line 118, in main
    args.func(args)
  File "/usr/local/lib/python3.6/dist-packages/allennlp/commands/evaluate.py", line 137, in evaluate_from_args
    instances = dataset_reader.read(evaluation_data_path)
  File "/usr/local/lib/python3.6/dist-packages/allennlp/data/dataset_readers/dataset_reader.py", line 234, in read
    instances = list(instances)
  File "/usr/local/lib/python3.6/dist-packages/tqdm/std.py", line 1130, in __iter__
    for obj in iterable:
  File "/nfs/strategyqa/src/data/dataset_readers/base/base_dataset_reader.py", line 77, in _read
    for instance in self._direct_read(file_path):
  File "/nfs/strategyqa/src/data/dataset_readers/strategy_qa_reader.py", line 102, in _direct_read
    instance = self._item_to_instance(self.json_to_item(json_obj))
  File "/nfs/strategyqa/src/data/dataset_readers/strategy_qa_reader.py", line 192, in _item_to_instance
    question, answer, decomposition, generated_decomposition, facts
  File "/nfs/strategyqa/src/data/dataset_readers/strategy_qa_reader.py", line 216, in text_to_instance
    facts=facts,
  File "/nfs/strategyqa/src/data/dataset_readers/strategy_qa_reader.py", line 283, in f
    if "unified" not in paragraphs:
TypeError: argument of type 'NoneType' is not iterable

Seeing that in strategy_qa_reader.py , def _ora_p() can return None and is then used by generate_context_from_paragraphs()
def _ora_p(self, decomposition, **kwargs):
        if decomposition is None or len(decomposition) == 0:
            return None

        num_of_matchings = len(decomposition[0]["evidence_per_annotator"])
        if num_of_matchings == 0:
            return None

        all_evidence_ids = [
            [step["evidence_per_annotator"][i] for step in decomposition]
            for i in range(num_of_matchings)
        ]
      ...
      ...
      ...

def generate_context_from_paragraphs(self):
        def f(**kwargs):
            print("get_paragrarph kwargs = ", kwargs)
            paragraphs = self.get_paragraphs(**kwargs) <---- when this returns None

            if "unified" not in paragraphs: <----- This causes error
                paragraphs.update({"unified": None})
                return None, paragraphs

I modified generate_context_from_paragraphs like so to it can update the dictionary and not cash when iterating over None
def generate_context_from_paragraphs(self):
        def f(**kwargs):
            print("get_paragrarph kwargs = ", kwargs)
            paragraphs = self.get_paragraphs(**kwargs)
            if paragraphs == None:             <-------- new
                paragraphs = {}                      <-------- new
            if "unified" not in paragraphs:

The script then runs! But the accuracy seems quite lower than the paper.
$ python3 run_scripts/evaluate.py --model pre-trained_models/8_STAR_ORA-P-D-last-step/8_STAR_ORA-P-D-last-step.tar.gz --data data/strategyqa/dev.json -g "1"
[lots of output, skipping]
INFO:allennlp.training.util:Iterating over dataset
0it [00:00, ?it/s]INFO:allennlp.data.samplers.bucket_batch_sampler:No sorting keys given; trying to guess a good one
INFO:allennlp.data.samplers.bucket_batch_sampler:Using ['tokens'] as the sorting keys
accuracy: 0.57, loss: 1.70 ||: : 115it [00:03, 36.13it/s]
INFO:allennlp.common.util:Metrics: {
  "accuracy": 0.5720524017467249,
  "loss": 1.7002240842188021
}
INFO:allennlp.commands.evaluate:Finished evaluating.

I suspect doing a workaround like I did isn't the correct fix. (• ◡•)
Could you point me in the right direction to get similar results to those reported?

Hi,

The code was missing a small piece, fixed it now.
You should run 8_STAR_ORA-P-D-last-step.tar.gz with data/strategy_qa/transformer_qa_ORA-P_dev_no_placeholders.json and not with data/strategy_qa/dev.json, because the model depends on using the last decomposition step without placeholders. I've updated the README file to mention it.

Thank you!

Fabulous

Using the following command I get 66.3% accuracy on DEV. Is that consistent with you @eladsegal ?
Seems a bit lower than the the test scores.
$ python3 run_scripts/evaluate.py --model pre-trained_models/8_STAR_ORA-P-D-last-step/8_STAR_ORA-P-D-last-step.tar.gz --data data/strategyqa/generated/transformer_qa_ORA-P_dev_no_placeholders.json -g "1"

INFO:allennlp.training.util:Iterating over dataset
0it [00:00, ?it/s]INFO:allennlp.data.samplers.bucket_batch_sampler:No sorting keys given; trying to guess a good one
INFO:allennlp.data.samplers.bucket_batch_sampler:Using ['tokens'] as the sorting keys
accuracy: 0.66, loss: 1.55 ||: : 115it [00:03, 35.74it/s]
INFO:allennlp.common.util:Metrics: {
  "accuracy": 0.6637554585152838,
  "loss": 1.5511564987638216
}
INFO:allennlp.commands.evaluate:Finished evaluating.

Also @eladsegal. (• ◡•)
Do you have dev scores for your linked models?
particularly for RoBERTa*_IR_D if elasticsearch isn't an option for one right now.

Sure, I've added metrics for all of our trained models in the README.
And you got a correct result, indeed there's a difference between the dev and test results.

Thank you for adding these!
Fascinating how RoBERTa* (non fine-tuned) gets 85.7% accuracy!
Any intuition why?
Noise in the dev set?

RoBERTa* is not evaluated on StrategyQA, but on BoolQ :)
It was mentioned in the notes below the models, but I've updated the README again so it will be more visible


Extended Beesweeper

Hi Dan and everybody else on the internet,
This is my version of the Beesweeper coding challenge as it was directly after the livestream. I added base functionality like better feedback, win condition, and the ability to place flags. But, I also gave the page a better design, added images, sound effects and a How To Play. (Sorry for making everything green, that's just my favorite color). I'm looking forward to seeing my contribution in a video soon!
kleines filmröllchen

This is amazing! I like to keep the code here matching what is in the video and then link outwards from a README.md file to variations. Can you put your code in another repo then do a pull request to add a README here? You can see some examples in other challenge folders. Thank you!!!
For more (this could probably use some work):
https://github.com/CodingTrain/Rainbow-Code/blob/master/CONTRIBUTING.md


Add option to use initial location as world origin

⚠ All PRs have to be done versus 'dev' branch, so be aware of that, or we'll close your issue ⚠
What kind of change does this PR introduce?

Adds the facility to use the initial GPS location as the WebGL/three.js/A-Frame world origin, as discussed in issue #499.
Can it be referenced to an Issue? If so what is the issue # ?
#499
How can we test it?

A new example initial-location-as-origin has been added to the A-Frame new-location-based examples. Try this, either on mobile or on desktop (with the latter, you will have to manually enter a latitude and longitude if your location cannot be determined). Click on any of the four visible boxes. These will show each box's world coordinates and either the x or z should have a value less than 200. Furthermore the camera position should be reported as x=0, y=1.6, z=0.
Summary

This is done via a new initialPositionAsOrigin property of gps-new-camera. It's false by default, so as not to break existing applications, but recommended to be set to true for new code.
The rationale for this is discussed in issue #499, namely that very large absolute values for world coordinates can cause precision problems. By setting the world origin to the initial GPS position (as was the case in the older location based components, as it happens) this problem is lessened.
Does this PR introduce a breaking change?
No - original behaviour is maintained if the property is not set.
Please TEST your PR before proposing it. Specify here what device you have used for tests, version of OS and version of Browser
Tested with Firefox on desktop (Linux) and Android 12/Chrome latest (Pixel 3), the latter using a real GPS location.
Other information
My proposal is to make this the default at version 3.5, but not yet to avoid breaking changes. Also at 3.5 the floating origin feature of #499 will be introduced, I think this will require a more significant rewrite.

Hi @nickw1 i will look into it very soon! thank you for this PR!

Hi @nickw1 i will look into it very soon! thank you for this PR!

@kalwalt thanks, there's also #507 too, I'd like to add these two to the next bugfix release if possible.

@nickw1 I hope in the next days to find the time to test this and #507...!

@nickw1 i tested your code with my Android device, and i can confirm that the boxes appear when you click on the "go" panel. I noticed a small issues, when you click on the colored box and the alert window is shown, i click the ok button to close it but it's closed and reappear again, and now if you push the ok button the window finally is closed.

as said in #507 see this comment i got this error:
aframe-ar.js:1 Uncaught ReferenceError: msg is not defined
    at i._displayError (aframe-ar.js:1:1638714)
    at Object.gpserror (aframe-ar.js:1:1636796)
    at _watchPositionId._watchPositionId.navigator.geolocation.watchPosition.enableHighAccuracy (aframe-ar.js:1:783100)

Hello @kalwalt thanks for testing. The issue you describe is something to do with the click event, though I am not sure what without investigating further. However, it is unrelated to this PR, it is only related to the demo. As long as you get world coordinates with low x and z values, the PR should be working.

Hello @kalwalt thanks for testing. The issue you describe is something to do with the click event, though I am not sure what without investigating further. However, it is unrelated to this PR, it is only related to the demo. As long as you get world coordinates with low x and z values, the PR should be working.

Agree with you, I will open an issue for that error i got, so we can keep track of it.

so if receive this message clicking on the red box:
Box clicked! Position is: 0 0 -117.0774072....(values omitted)

does it is correct in your opinion?

Yes, looks fine.

OK done. Once I've merged this shall I make a new release?

OK done. @kalwalt Once I've merged this shall I make a new release?

yes, can you update also artoolkit5-js package to the new version before the new release?

So 0.3.0 ?

So 0.3.0 ?
yes exactly this the version where i added the type definitions, it shouldn't create any issue. i have already tested in my local branch. thank you! 🙂

So 0.3.0 ?

yes exactly this is the version where i added the type definitions, it shouldn't create any issue. i have already tested in my local branch. thank you! slightly_smiling_face

OK great, will try and make a release tonight.


Cli tool --output path has to already exist and is counter-intuative

Hello
I just had my first experience with the SwashBuckle CLI tool and ran into an issue.
I copied the basic example for generating a swagger.json at build time.
<Target Name="SwaggerToFile" AfterTargets="AfterBuild">
    <Exec Command="swagger tofile --host http://example.com --output wwwroot/api-docs/v1/swagger.json "$(TargetPath)" v1" EnvironmentVariables="ASPNETCORE_ENVIRONMENT=Development" />
</Target>

It appears the --output argument directory we can specify has to already exist. This is very counter intuative for an optional argument of a cli tool that generates new content.
Error:
SwasbuckleCliRepro -> D:\repos\SwasbuckleCliRepro\bin\Debug\net6.0\SwasbuckleCliRepro.dll
  Unhandled exception. System.IO.DirectoryNotFoundException: Could not find a part of the path 'D:\repos\SwasbuckleCliRepro\wwwroot\api-docs\v1\swagger.json'.
     at Microsoft.Win32.SafeHandles.SafeFileHandle.CreateFile(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options)
     at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize)
     at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize)
     at System.IO.Strategies.FileStreamHelpers.ChooseStrategyCore(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize)
     at System.IO.Strategies.FileStreamHelpers.ChooseStrategy(FileStream fileStream, String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, Int64 preallocationSize)
     at System.IO.StreamWriter.ValidateArgsAndOpenPath(String path, Boolean append, Encoding encoding, Int32 bufferSize)
     at System.IO.File.CreateText(String path)
     at Swashbuckle.AspNetCore.Cli.Program.<>c.<Main>b__0_4(IDictionary`2 namedArgs) in C:\projects\ahoy\src\Swashbuckle.AspNetCore.Cli\Program.cs:line 100
     at Swashbuckle.AspNetCore.Cli.CommandRunner.Run(IEnumerable`1 args) in C:\projects\ahoy\src\Swashbuckle.AspNetCore.Cli\CommandRunner.cs:line 68
D:\repos\SwasbuckleCliRepro\SwasbuckleCliRepro.csproj(14,5): error MSB3073: The command "swagger tofile --host http://example.com --output wwwroot/api-docs/v1/swagger.json "D:\repos\SwasbuckleCliRepro\bin\Debug\net6.0\SwasbuckleCliRepro.dll" v1" exited with code -532462766.
D:\repos\SwasbuckleCliRepro\SwasbuckleCliRepro.csproj(14,5): error MSB3073: The command "swagger tofile --host http://example.com --output wwwroot/api-docs/v1/swagger.json "D:\repos\SwasbuckleCliRepro\bin\Debug\net6.0\SwasbuckleCliRepro.dll" v1" exiBuild FAILED.

Request: Add a "Create directory if it does not exist" step to make cli tool more intuative and add ability to insert variables into output path, like the current ci environment, or the api version.
Would you accept a PR for this?
Minimal repro

I think this is a good suggestion, but I want to also say thanks for posting your example showing how to escape the first - as -. I was struggling to find a convenient way to comment out that line of my project file, as -- is invalid in an XML comment, and the CLI doesn't take single letter flag variants (eg. -o in lieu of --output).  Cheers!

I just spent half an hour trying to sort out the CLI tool parameters. Here's the usage line:
Usage: dotnet swagger tofile [options] [startupassembly] [swaggerdoc]
In usage info, brackets usually indicate optional arguments. Minimally, when a required argument is not provided, the output should indicate at least the first required missing argument. Useful info in this Q&A:
https://stackoverflow.com/questions/9725675/is-there-a-standard-format-for-command-line-shell-help-text
Another pain is that the options item indicates the following:
--output:  relative path where the Swagger will be output, defaults to stdout
I have found no way to make it work if --output is missing. Additionally the value is that of the path and file name.
Hopefully the following working example helps others:
dotnet swagger tofile --output .\MyWebAPI.dll v1
Where v1 is the name of the swagger document as seen in the setup via AddSwaggerGen.

https://github.com/domaindrivendev/Swashbuckle.AspNetCore/issues/2667#issuecomment-2053980759

Is there any movement on addressing this issue? It's totally unintuitive to supply an output path and then be told it doesn't exist :-(

See the linked comment - a pull request would be welcome.


Enhance the clusteradm to support installing klusterlet in the hosed mode

When joining as cluster to a hub, user is able to set the install mode of the klusterlet to Hosted with command similar to
clusteradm join  …. --mode hosted --managed-cluster-kubeconfig kubeconfig

/close


v6 Image plugin

When I insert an image on the page, it's not confined to it's container, even on the bootstrap template, where I know the default is max-width: 100%.

I think this is fixed in RC1 (at least for Bootstrap).  I am going to keep this open for now.