Bhe/bugfix/tup 22575 upgrade org eclipse jdt core

fix(TUP-22575) Upgrade hadoop-core from 1.0.0 to 1.2.1,org.eclipse.jdt:core:jar:3.1.1 to org.eclipse.jdt:core:jar:3.3.0-v_771,com.cloudera.api:cloudera-manager-api:jar:5.10.0 to 6.2.0
What is the current behavior? (You can also link to an open issue here)
No behavior changes
What is the new behavior?
No behavior changes
Please check if the PR fulfills these requirements

[x] The commit message follows Talend standard
[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features) ?
[ ] The code coverage on new code >75%
[x] The new code does not introduce new technical issues (sonar / eslint)

What kind of change does this PR introduce?

[x] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build / CI related changes
[ ] Other... Please describe:

Does this PR introduce a breaking change?

[ ] Yes
[x] No

If this PR contains a breaking change, please describe the impact and migration path for existing applications: ...
Other information:

Pull request test result: success :star:
Sonar results

Note, this PR should be merged after https://github.com/Talend/tbd-studio-se/pull/1082, I will resolve comments of https://github.com/Talend/tbd-studio-se/pull/1082 firstly. Thanks

Pull request test result: success :star:
Sonar results


logging is nearly non-existent in the API server

web servers provide us with access logs by default.  beyond that, logging is very sparse in our API server.
it looks like these are the only explicit log emissions, found using the command grep -rn log src/server/ | grep \.py | grep -v src/server/covidcast_issues_migration :

a parameterized version of SQL queries upon execution
a stale metacache
an error in getting/creating a db connection
an error (presumably) in rendering a result set
an error parsing a date

in addition to these, we can and should log:

web request parameters and/or SQL queries in a way that can be deduplicated and profiled more easily than the current
other information for worthwhile metrics: rows returned, bytes returned, http connection time, db query time, various error codes/reasons, ... (the possibilities are vast)

we can add this info to dashboards and alerting, and it is a good way for us to get useful, actionable information in the short and long term
here is one approach to doing so: #757

there are also likely many places we can trap and report on unexpected inputs, edge cases, exceptions, or other errors.

also: all log messages should include process/thread/worker id so we can trace the execution that each does throughout the request/response process.

@melange396 it looks like this is currently implemented for most logging messages - e.g.
[2022-11-10 13:39:56 -0500] [10] [INFO] Booting worker with pid: 10
[2022-11-10 13:39:56 -0500] [11] [INFO] Booting worker with pid: 11
[2022-11-10 13:39:56 -0500] [12] [INFO] Booting worker with pid: 12
[2022-11-10 13:39:56 -0500] [13] [INFO] Booting worker with pid: 13
[2022-11-10 13:39:56 -0500] [14] [INFO] Booting worker with pid: 14
[2022-11-10 13:39:56 -0500] [29] [INFO] Booting worker with pid: 29
[2022-11-10 13:39:56 -0500] [30] [INFO] Booting worker with pid: 30
[2022-11-10 13:39:56 -0500] [38] [INFO] Booting worker with pid: 38
...
[2022-11-10 13:40:22 -0500] [38] [INFO] Received API request: {'method': 'GET', 'path': '/epidata/api.php?data_source=src&signals=sig&time_type=day&geo_type=county&time_values=20000101&geo_value=11111&source=covidcast', 'args': ImmutableMultiDict([('data_source', 'src'), ('signals', 'sig'), ('time_type', 'day'), ('geo_type', 'county'), ('time_values', '20000101'), ('geo_value', '11111'), ('source', 'covidcast')])}
[2022-11-10 13:40:22 -0500] [10] [INFO] Optimizing list of time values [20000101]...
[2022-11-10 13:40:22 -0500] [11] [INFO] Received API request: {'method': 'GET', 'path': '/epidata/api.php?data_source=src&signals=sig&time_type=day&geo_type=county&time_values=20000101&geo_value=00000&source=covidcast', 'args': ImmutableMultiDict([('data_source', 'src'), ('signals', 'sig'), ('time_type', 'day'), ('geo_type', 'county'), ('time_values', '20000101'), ('geo_value', '00000'), ('source', 'covidcast')])}

but these HTTP requests are logged in a different format:
<IP_ADDRESS> - - [10/Nov/2022:13:40:22 -0500] "GET /epidata/api.php HTTP/1.1" 200 41 "-" "delphi_epidata/0.4.0"
<IP_ADDRESS> - - [10/Nov/2022:13:40:22 -0500] "- - HTTP/1.0" 0 0 "-" "-"

I'll investigate why that's the case.


Cannot read property '0' of undefined (Executed Test Event CloudWatch)

Hi,
I've everything set-up correctly and before using the lambda function, I wanted to test it by creating a new test (from AWS Console) and selected the CloudWatch template, once executed it gives this error:
Cannot read property '0' of undefined
    at processEvent (/var/task/index.js:321:43)
    at exports.handler (/var/task/index.js:369:5)

Any clue?

If you're using the Lambda test function, and not providing a valid payload, that would be the problem. You can take an event's JSON out of the test suite and paste it directly into the Lambda test payload editor.

Hi @creichert ,
Thank for the reply. With real logs it's working fine (tested), but as I previously said, if I test with the CloudWatch template (provided by AWS) it failed to work (I guess the payload it's different but since the template it's a basic CloudWatch I thought it would work and expected to receive a Generic Notification.
I tried to check the logs but it doesn't exist. On Log output there is a link to access the logs "Click here to view the CloudWatch log group.", but when I click it opens the CloudWatch logs page and says:
There was an error loading Log Streams. Please try again by refreshing this page.
Anyways, checked again the CloudWatch template, and it has this payload:
{
  "awslogs": {
    "data": "H4sIAAAAAAAAAHWPwQqCQBCGX0Xm7EFtK+smZBEUgXoLCdMhFtKV3akI8d0bLYmibvPPN3wz00CJxmQnTO41whwWQRIctmEcB6sQbFC3CjW3XW8kxpOpP+OC22d1Wml1qZkQGtoMsScxaczKN3plG8zlaHIta5KqWsozoTYw3/djzwhpLwivWFGHGpAFe7DL68JlBUk+l7KSN7tCOEJ4M3/qOI49vMHj+zCKdlFqLaU2ZHV2a4Ct/an0/ivdX8oYc1UVX860fQDQiMdxRQEAAA=="
  }
}

So I guess that's the problem, is not a complete CloudWatch log.
So I think everything is working good so far. Thanks again.


The digit-five dataset is no longer exist

Hi Trgtuan, the Digit-five dataset was provided by Peng et al. and unfortunately the link no longer exists. Maybe you can try this one: https://wjdcloud.blob.core.windows.net/dataset/dg5.tar.gz from this related issue.

Thanks @jonas-lippl


[Aio] Client unary unary interceptor

Implement the unary interceptor for the client-side. Interceptors
can be now installed by passing them as a new parameter of the Channel constructor or
by giving them as part of the insecure_channel function.
Interceptors are executed within the RPC Asyncio task, receiving as a
response the response body sent by the downstream layer implemented in
Cython.
Main differences with the synchronous implementation.
How interceptors are added
There is a significative difference in how interceptors are being added in the Aio package and the interface provided by the synchronous package, while the last one allows composition the Aio package publishes a very naive implementation which only accepts the interceptors as a new constructor keyword, which can be implicitly given by using the insecure_channel primitive.
For example, the following snippet shows how interceptors would need to be installed from now on when Aio module is used.
channel = insecure_channel(..., interceptors=[MyInterceptor()])

Object returned by a continuation
In Aio the object returned by the continuation is the result of the unary call made by using the Cython interfaces. Right now only bytes but in the future most likely a more complex object with more attributes - for example for returning the metadata related to an object.
Following code shows how the interceptor handle takes this into consideration and also uses the proper except statements for catch any known exception
async def intercept_unary_unary(self, continuation, client_call_details,
                                request):
    status_code = grpc.StatusCode.OK
    try:
        return await continuation(client_call_details, request)
    except aio.AioRpcError as aio_rpc_error:
        status_code = aio_rpc_error.code()
        raise
    except asyncio.CancelledError:
        status_code = grpc.StatusCode.CANCELLED
        raise
    finally:
        self.status_codes_observed[status_code] += 1

Note: these differences - if are accepted - would need to be added in the gRFC [1]
[1] https://github.com/grpc/proposal/pull/155

@lidizheng regarding the question

Can we support the existing version of the intercepting API as well?

Is this a must? I would like to not have to implement all of the weird logic behind the _Channel object.
But in any case, I would advocate for having for now only one way of adding interceptors, the old one or the new one but not both right now.

@pfreixes If we take a closer look at the API grpc.intercept_channel. It is similar to what we have here. I think you are right about implementing one method first. The old version API can wait for later if any one needs it.

@lidizheng ready for a second review, I'm still sticking to the plan on having the continuation returning what is being returned by the Cython code, which is right now the payload only, due to the cost of using tasks.

@gnossen PTAL.

Closing in favor of this new one https://github.com/grpc/grpc/pull/21455


Add typing to admin web routes

This is here as a test to see if we can feasibly add typing to flask code
Pull request checklist:

[x] CHANGELOG.md was updated, if applicable
[x] Documentation in docs/ or install-docs/ was updated, if applicable
[ ] I have tested all changes

Tested:
Configuring/enabling modules
Configuring/enabling/creating/deleting banphrases
Configuring/creating/deleting commands
Configuring/creating/deleting timers
Configuring/creating/deleting playsounds


Dash app crashes when you clear selection

Dash app crashes when you clear the select list:

[2020-11-09 11:05:43,804] ERROR in app: Exception on /_dash-update-component [POST]
Traceback (most recent call last):
  File "/home/dblank/.local/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/dblank/.local/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/dblank/.local/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/dblank/.local/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/dblank/.local/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/dblank/.local/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/dblank/.local/lib/python3.8/site-packages/dash/dash.py", line 1059, in dispatch
    response.set_data(func(*args, outputs_list=outputs_list))
  File "/home/dblank/.local/lib/python3.8/site-packages/dash/dash.py", line 994, in add_context
    output_value = func(*args, **kwargs)  # %% callback invoked %%
  File "../codecarbon/viz/carbonboard.py", line 77, in update_project_data
    project_summary = data.get_project_summary(project_data.data)
  File "/home/dblank/comet/code-carbon/codecarbon/viz/data.py", line 24, in get_project_summary
    last_run = project_data[-1]
IndexError: list index out of range

Is the csv file empty ? @dsblank

No, it is not empty.

Unable to replicate this on my end.


Update CONTRIBUTORS.md

As requested in issue #48
Changes made:

Added the details of two contributors to CONTRIBUTORS.md
Updated the icons to the best of my knowledge
Copied the table to README.md

For some reason, the table remained at the end though I moved the code appropriately... So I moved the heading down instead. Please take a look @swapnilsparsh

For some reason, the table remained at the end though I moved the code appropriately... So I moved the heading down instead. Please take a look @swapnilsparsh

For some reason, the table remained at the end though I moved the code appropriately... So I moved the heading down instead. Please take a look @swapnilsparsh

Don't move the whole contributors section to the end of README.md file
Keep the Contributors sections where it was earlier and there only place your table.

For some reason, the table remained at the end though I moved the code appropriately... So I moved the heading down instead. Please take a look @swapnilsparsh

Don't move the whole contributors section to the end of README.md file
Keep the Contributors sections where it was earlier and there only place your table.

For some reason, the table remained at the end though I moved the code appropriately... So I moved the heading down instead. Please take a look @swapnilsparsh

Don't move the whole contributors section to the end of README.md file
Keep the Contributors sections where it was earlier and there only place your table.

Done 👍

For some reason, the table remained at the end though I moved the code appropriately... So I moved the heading down instead. Please take a look @swapnilsparsh

Don't move the whole contributors section to the end of README.md file
Keep the Contributors sections where it was earlier and there only place your table.

Done 👍

Thanks a bunch!

Thanks a bunch!


Update rack and rails for multiple CVEs

Update rack to <IP_ADDRESS> for CVE-2023-27539
Update rails to <IP_ADDRESS> for CVE-2023-23913 and CVE-2023-28120

@jrafanie Opening this on master first, then will cherry-pick this into #22411

@Fryguy A conflict occurred during the backport of this pull request to petrosian.
If this pull request is based on another pull request that has not been marked for backport, add the appropriate labels to the other pull request. Otherwise, please create a new pull request direct to the petrosian branch in order to resolve this.
Conflict details:

This was a manual PR and was merged in #22411


Bdir-dshot 300 error 100%

Hi, I'm using dshot300 with Bidir on. But I got 100% error in betaflight and RPM 0 all the time. I've tested it on a stm32f3 and it is working. hence the ESC is working correctly. is this normal?

what type of esc are you using and what firmware is loaded to it?

ESP32-S3
latest-esp-fc master, unchanged
G-h-20, 16,9, https://github.com/JazzMaverick/BLHeli/tree/JazzMaverick-patch-1?tab=readme-ov-file

@sysytwl Why JazzMaverick? this firmware is not maintained for very long time. So far I've tested combination of ESP32 + Bluejay. For S3 I'm in process of completiing hardware ATM So I'll be able to verify in several days.

ok, thx for the info. I'll check that.

hi, I tried with Bluejay. It seems to have the same issue.

@sysytwl in develop branch there are fixes for dshot-bidir, now rpm should be shown on ESP32-S3. If you are able to verify, that would be great.

it's working. thx a lot.


Don't throw when returned value is not a promise

@facebook-github-bot import

Are you trolling me? You mentioned coffeescript in a code comment in a project I work on? What!?

Thanks for importing. If you are an FB employee go to Phabricator to review.


FIX HttpCrawler: add site map include parameter when creating HttpQueueP...

For previous discussion: https://github.com/Norconex/collector-http/pull/65
Site map is not always read by collector-http 2.0.2

FYI, I merged, but then changed it to always execute the sitemap stage based on your feedback (and it will decide whether to actually do something or not based on the configuration settings).


Running Humbug on Humbug

I get lots of errors running Humbug on Humbug. Errors are connected with phpunit-extensions installed some time ago.
PHP Fatal error:  Uncaught exception 'RuntimeException' with message 'FastestFirstFilter has encountered an unlogged test suite which cannot be sorted' in /home/rolen/php/humbug/vendor/padraic/phpunit-extensions/src/Humbug/Phpunit/Filter/TestSuite/FastestFirstFilter.php:30
Stack trace:
#0 [internal function]: Humbug\Phpunit\Filter\TestSuite\FastestFirstFilter->Humbug\Phpunit\Filter\TestSuite\{closure}(Object(PHPUnit_Framework_TestSuite), Object(PHPUnit_Framework_TestSuite))
#1 /home/rolen/php/humbug/vendor/padraic/phpunit-extensions/src/Humbug/Phpunit/Filter/TestSuite/FastestFirstFilter.php(41): usort(Array, Object(Closure))
#2 /home/rolen/php/humbug/vendor/padraic/phpunit-extensions/src/Humbug/Phpunit/Listener/FilterListener.php(61): Humbug\Phpunit\Filter\TestSuite\FastestFirstFilter->filter(Array)
#3 /home/rolen/php/humbug/vendor/padraic/phpunit-extensions/src/Humbug/Phpunit/Listener/FilterListener.php(47): Humbug\Phpunit\Listener\FilterListener->filterSuites(Array)
#4 /home/rolen/php/humbug/vendor/phpunit/phpunit/src/Framew in /home/rolen/php/humbug/vendor/padraic/phpunit-extensions/src/Humbug/Phpunit/Filter/TestSuite/FastestFirstFilter.php on line 30

This error occurs 214 times. See https://gist.github.com/rollenes/fd4930184ad7e2375de5#file-humbug-issue-txt
FastestFirstFilter is not able to find time logs for specific tests in phpunit.times.humbug.json.
After some analysis I found that this file(phpunit.times.humbug.json) is changing while running humbug.
I run tail on phpunit.times.humbug.json. See https://gist.github.com/rollenes/fd4930184ad7e2375de5#file-humbug-times-tail-log
It appears to truncate file many times while running.
So I simply run phpunit on humbug test harness and effect was the same. It overrides phpunit.times.humbug.json several times.
See tail below:
rolen@rollen:/tmp $ tail -f phpunit.times.humbug.json
{
    "suites": [],
    "tests": {
        "MM1_MathTest": [
            {
                "title": "testAdds",
                "time": 0.010890007019043
            }
        ]
    }
}.0039820671081543
            }
        ]
    }
}tail: phpunit.times.humbug.json: file truncated
tail: phpunit.times.humbug.json: file truncated
{
    "suites": [],
    "tests": {
        "ExceptionTest": [
            {
                "title": "testSomeException",
                "time": 0.0022640228271484
            }
        ]
    }
}tail: phpunit.times.humbug.json: file truncated

So each time tests are run in humbug as a background phpunit process it truncates file and pushes new content to phpunit.times.humbug.json.
I tracked those tests under Humbug\Test\Adapter\PhpunitTest. Skipping this tests made possible to run phpunit without affecting phpunit.times.humbug.json.
Running humbug then also did not affected this file. And all errors disappeared.
But I didn`t want to skip them...
After some another analysis of Humbug\Test\Adapter\PhpunitTest I found that those test writes all of its contents to cacheDirectory.
So I changed it, and unskipped those tests.
Phpunit run didnt affects phpunit.times.humbug.json .So problem should disappear...
But I still get the same errors running Humbug on Humbug:
PHP Fatal error:  Uncaught exception 'RuntimeException' with message 'FastestFirstFilter has encountered an unlogged test suite which cannot be sorted' in /home/rolen/php/humbug/vendor/padraic/phpunit-extensions/src/Humbug/Phpunit/Filter/TestSuite/FastestFirstFilter.php:30

Also tail of phpunit.times.humbug.json behaved in the same way(truncated and replaced content).
Digging dipper...
First some cleanups - I left those new cache folders in test suites so I thought it would be nice to remove it in tearDown.
PhpUnit failed:
rmdir(/tmp/8583355): Directory not empty

rolen@rollen:~/php/humbug (master) $ ls /tmp/5117136
coverage.humbug.php  coverage.humbug.txt  phpunit.humbug.xml

So there are some files in this place after running phpunit. This is some kind of side effect, so I remove this files.
PhpUnit run: passes
Humbug on humbug: no changes...
Still lots of errors with FastestFirstFilter and tail of phpunit.times.humbug.json reports to be truncated many times.
Lets debug then, and trace changes in files with side effects mentioned before:

phpunit.humbug.xml
phpunit.times.humbug.json

with debugging points after initial check is done and after each mutant process is executed. No results as those processes are run in background and from debugging tools its hard to jump into new paraller process.
I put some logs here and there...
It turns out that one of mutant was creating phpunit.humbug.xml with content bellow:
<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://schema.phpunit.de/4.4/phpunit.xsd" backupGlobals="false" backupStaticAttributes="false" colors="true" convertErrorsToExceptions="true" convertNoticesToExceptions="true" convertWarningsToExceptions="true" processIsolation="false" stopOnFailure="false" syntaxCheck="false" bootstrap="/tmp/humbug.phpunit.bootstrap.php" cacheTokens="false">

    <testsuites>
        <testsuite name="Test Suite">

        <directory>/home/rolen/php/humbug/tests/Adapter/_files/phpunit</directory></testsuite>
    </testsuites>

<listeners><listener class="\MyBuilder\PhpunitAccelerator\TestListener"><arguments><boolean>true</boolean></arguments></listener><listener class="\Humbug\Phpunit\Listener\TimeCollectorListener"><arguments><object class="\Humbug\Phpunit\Logger\JsonLogger"><arguments><string>/tmp/1514814/phpunit.times.humbug.json</string></arguments></object></arguments></listener></listeners><logging><log type="coverage-php" target="/tmp/1514814/coverage.humbug.php"/><log type="coverage-text" target="/tmp/1514814/coverage.humbug.txt"/></logging><filter><whitelist/></filter></phpunit>

This mutant was not written in humbuglog.txt:
XmlConfiguration:
-        self::$dom->preserveWhiteSpace = false;
+        self::$dom->preserveWhiteSpace = true;

Running phpunit against phpunit.humbug.xml produced:
rolen@rollen:~/php/humbug (master) $ vendor/bin/phpunit -c phpunit.humbug.xml
PHP Notice:  Trying to get property of non-object in /home/rolen/php/humbug/vendor/phpunit/phpunit/src/Util/XML.php on line 231
PHP Stack trace:
PHP   1. {main}() /home/rolen/php/humbug/vendor/phpunit/phpunit/phpunit:0
PHP   2. PHPUnit_TextUI_Command::main($exit = *uninitialized*) /home/rolen/php/humbug/vendor/phpunit/phpunit/phpunit:36
PHP   3. PHPUnit_TextUI_Command->run($argv = *uninitialized*, $exit = *uninitialized*) /home/rolen/php/humbug/vendor/phpunit/phpunit/src/TextUI/Command.php:103
PHP   4. PHPUnit_TextUI_TestRunner->doRun($suite = *uninitialized*, $arguments = *uninitialized*) /home/rolen/php/humbug/vendor/phpunit/phpunit/src/TextUI/Command.php:151
PHP   5. PHPUnit_TextUI_TestRunner->handleConfiguration($arguments = *uninitialized*) /home/rolen/php/humbug/vendor/phpunit/phpunit/src/TextUI/TestRunner.php:153
PHP   6. PHPUnit_Util_Configuration->getListenerConfiguration() /home/rolen/php/humbug/vendor/phpunit/phpunit/src/TextUI/TestRunner.php:687
PHP   7. PHPUnit_Util_XML::xmlToVariable($element = *uninitialized*) /home/rolen/php/humbug/vendor/phpunit/phpunit/src/Util/Configuration.php:339
PHP Warning:  Invalid argument supplied for foreach() in /home/rolen/php/humbug/vendor/phpunit/phpunit/src/Util/XML.php on line 234
PHP Stack trace:
PHP   1. {main}() /home/rolen/php/humbug/vendor/phpunit/phpunit/phpunit:0
PHP   2. PHPUnit_TextUI_Command::main($exit = *uninitialized*) /home/rolen/php/humbug/vendor/phpunit/phpunit/phpunit:36
PHP   3. PHPUnit_TextUI_Command->run($argv = *uninitialized*, $exit = *uninitialized*) /home/rolen/php/humbug/vendor/phpunit/phpunit/src/TextUI/Command.php:103
PHP   4. PHPUnit_TextUI_TestRunner->doRun($suite = *uninitialized*, $arguments = *uninitialized*) /home/rolen/php/humbug/vendor/phpunit/phpunit/src/TextUI/Command.php:151
PHP   5. PHPUnit_TextUI_TestRunner->handleConfiguration($arguments = *uninitialized*) /home/rolen/php/humbug/vendor/phpunit/phpunit/src/TextUI/TestRunner.php:153
PHP   6. PHPUnit_Util_Configuration->getListenerConfiguration() /home/rolen/php/humbug/vendor/phpunit/phpunit/src/TextUI/TestRunner.php:687
PHP   7. PHPUnit_Util_XML::xmlToVariable($element = *uninitialized*) /home/rolen/php/humbug/vendor/phpunit/phpunit/src/Util/Configuration.php:339
PHPUnit 4.4.5 by Sebastian Bergmann.

Configuration read from /home/rolen/php/humbug/phpunit.humbug.xml

.

Time: 10.25 seconds, Memory: 6.50Mb

OK (1 test, 1 assertion)

Generating code coverage report in PHP format ... done

and /tmp/phpunit.times.humbug.json was affected with content:
{
    "suites": {
        "MM1_MathTest": 0.012728929519653
    },
    "tests": {
        "MM1_MathTest": [
            {
                "title": "testAdds",
                "time": 0.012728929519653
            }
        ]
    }
}

Eureka!
All because of JsonLogger::__construct(), If there is no $target given it will write into /tmp/phpunit.times.humbug.json.
I changed logic in JsonLogger::__construct() and revert this mutant in codebase.
Phpunit: failed with fatal error.
Humbug on humbug: no errors :) And phpunit.times.humbug.json was not changed since first run.

To resolove it merge #99 is required.
And update of phpunit-extensions by https://github.com/padraic/phpunit-extensions/pull/2 is required.

Merged! Appreciate the debugging patience :). If this resolves the problem, I'll let you close the issue.

After merge and update of phpunit-extensions via composer update I didn`t noticed this problem again.


Scope not clear

There should be some kind of explicit statement of scope. I am not sure from the text how wide it applies.
For example, suppose Alice (a white female) and Bob (a black male) are members of the community of project X, which uses this CoC.
Further suppose that Alice and Bob are also both members of some other community, such as a forum for an online game they both play. That community has no connection to project X, other than some members of it are also members of the X community.
In that forum, discussion is often heated and nasty. Suppose Alice makes racist remarks about Bob in that forum, or Bob makes sexist remarks about Alice in that forum.
Would those remarks in that outside forum be violations of the Project X CoC, since Alice and Bob are both X members? Or are they out of scope because they did not take place in X forums, mailing lists, IRC, etc.?

That is a difficult question, as the line between personal life and project public relations is very blurry and it is hard to find a border which satisfies everyone.

Thanks for your feedback. See https://github.com/todogroup/opencodeofconduct/issues/84.


Fix Tooltips requiring UnityEditor

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
Bug fix

What is the current behavior? (You can also link to an open issue here)
Tooltips requires a UnityEditor extension, which means it can't be built into an app. The extension used is the PrefabUtility to instantiate prefabs.

What is the new behavior? (if this is a feature change)
Now uses Instantiate instead of InstantiatePrefab, removing the need for UnityEditor.

Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)
No

Other information:
Fixes #258

I know but I opted to let you commit it since it's your branch. I think you should commit it so others doesn't have to do it

I believe that file is part of the gitignore, and can't be pushed


json schema defined in local file configured via json.schema in user settings.json is ignored

VSCode Version: 1.36 (also not working on the insider build)
OS Version: Win 10 64bit

I am trying to configure a json schema that is not ready for sharing publicly via the user settings.json to avoid having to configure it in every workspace on my machine.
Same configuration works in a workspace .vscode/settings.json though.
Steps to Reproduce:

Follow instructions to configure json schemas in user settings.json
But instead of pointing to a schema hosted online, point to a local file via an absolute file url
Json file that matches the file name pattern does not get validated
Move the same json.schemas configuration to the workspace .vscode/settings.json and the same Json file immediately starts being validated by the schema.

Does this issue occur when all extensions are disabled?: Yes
Here is how my json.schemas configuration looks like:
    "json.schemas": [
        {
            "fileMatch": [
                "*.myConfiguration.json"
            ],
            "url": "file:///C:/directory/myConfiguration.json"
        }
    ]

And I am naturally testing it on a file named test.myConfiguration.json to match the fileMatch pattern.

I just tried this setup and it worked for me. Not sure what went wrong in your case.

A way to debug this is by looking at the json language server log.
Define `"json.trace.server": "verbose" in the settings and check the logs.


Create Weather tool.python

To create a tool that generates weather reports, we'll need to utilize a weather API to fetch weather data and present it to the user. In this example, I'll use the OpenWeatherMap API to fetch weather data for a given location. You'll need to sign up for an API key from OpenWeatherMap (https://openweathermap.org/api) to use their service. #hactoberfest2023 #hacktoberfest-accepted
🛠 Fixes issue #
📇 Changes proposed
✅ Check List (Mark all the applicable boxes)

[ ] My code follows the code style of this project.
[ ] This PR does not contain plagiarized content.
[ ] This PR contains changes in the documentation
[ ] This PR contains changes in the Code of the project
[ ] The title of my pull request is a short description of the requested changes.

📄 Note to reviewers

📷 Screenshots

Not Required @shivangi241103. I am closing this PR


Component should accept className and styles should all be obj OR string

Container style, all the other styles... need to let custom css module style names go in there in place of style objects. Also, the main component should accept standard Bootstrap 'className' prop.

@L5eoneill, all the className accept the string or a function which generate class name dynamically. and the style only accept object which follow jsx principle
<BootstrapTable insertRow exportCSV data={ products }
        tableStyle={ { border: '#0000FF 2.5px solid' } }
        containerStyle={ { border: '#FFBB73 2.5px solid' } }
        trClassName='tr-string-example'
>
          <TableHeaderColumn dataField='id' isKey={ true }>Product ID</TableHeaderColumn>
          <TableHeaderColumn dataField='name'>Product Name</TableHeaderColumn>
          <TableHeaderColumn dataField='price'>Product Price</TableHeaderColumn>
      </BootstrapTable>

BTW, I'm not sure if I've answered your question, feel free to discuss with me :)
thanks

Yeah, doesn't completely address the issue, which is that many of the elements your component creates don't receive class names. Accepting a style object just isn't good enough. We get:
react-bs-table-container
react-bs-table
react-bs-container-header
react-bs-container-body
From your docs, none of these can take on a custom css class name, so if you're using css-modules, you have to work really hard to override even the smallest of default bootstrap stylings, using tricks like !important and :global(.react-bs-table) and such, which are frowned upon.

@L5eoneill, ok,  currently, tableHeaderClass and tableBodyClass only apply class name to header and body table. Maybe  I can support another props for supporting to apply class name to following elements:
react-bs-table-container
react-bs-table
react-bs-container-header
react-bs-container-body

Yes, that'd be excellent. Thanks.

use containerClass to apply class to react-bs-table-container
use tableContainerClass to apply class to react-bs-table
use headerContainerClass to apply class to react-bs-container-header
use bodyContainerClass to apply class to react-bs-container-body
Released on v2.4.4

there's one thing to notice, on v2.4.4 I move the css files from css to dist folder, remember to update the css path. I apologize for it.


0.4.1 (from PyPI) is broken: No such file or directory:  ... /README.md

It attempts to open a non-existent file:
===>  Configuring for py38-pyked-0.4.1
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "setup.py", line 11, in <module>
    with open(path.join(here, 'README.md')) as readme_file:
  File "/usr/local/lib/python3.8/codecs.py", line 905, in open
    file = builtins.open(filename, mode, buffering)
FileNotFoundError: [Errno 2] No such file or directory: '/disk-samsung/freebsd-ports/science/py-pyked/work-py38/pyked-0.4.1/README.md'
*** Error code 1

Yep, probably a problem with the MANIFEST.in file, which would need to include the README.md file. Please feel free to submit a pull request to fix it.

The current GH master doesn't have this problem.


Intermittent Error: Collection is not initialized!

On the application start I connect to the MongoDB instance and run the initialize function before doing some migrations.
But some times I get this error Error: Collection is not initialized! other times I don't. It's intermittent.
What are the main reasons for this error to happen, even though my collection was initialized?

Hi!
This error could happen if you are trying to access a collection before it has been initialised. A possibility is that you have a race condition in your code. Could you please share a snapshot of your code?

Hey, that makes sense. But it proved very difficult for me to detect what's causing the race condition.
I'm using Next.js and on every API call I'm calling and awaiting this connectToDb async function. To avoid race condition, I create a global that keeps the Promise of the connection so if any other API get's called at the same time, it should await for the Promise instead of calling the connect function again but maybe I'm doing something wrong.
This is the code if you want to check it out:
https://gist.github.com/italodeandra/c45c1fb419c8671c9db9da5376d8bef9

I added some comments to your gist

It's been ages and I still have this problem. It doesn't happen with mongoose or the native driver. Is there any other suggestion of what might be causing it?

Can you provide a minimal reproduction case without next.js in the mix?

I'll try, but this issue seems to be specific to how Next.js create new instances of the same file (APIs) when you change them. It might be something related to race condition because that only happens on the start. And it is also intermittent. I usually just restart the app two or three times until it works.

Having the same problem with papr in a remix app.
It's even worse in remix as the default setup has no easy way to do any kind of setup before the app starts and remix currently re-requires the whole app on every request.
This gets better if one opts into unstable remix features, but still occurs in "dev-mode" when the server restarts (every time).
Other database libraries get around this by queuing commands even tho the database connection is not ready yet. I think papr should also do this. Lots of dev headaches can be avoided that way.
I implemented my own hacky solution to this by wrapping all my papr models with a Proxy. You could try this @italodeandra
papr.ts (file that handles connection)
import { MongoClient } from "mongodb";
import Papr from "papr";

const papr = new Papr();
export let client: MongoClient;

const connection = (async () => {
  console.log("connecting to db");
  client = await MongoClient.connect(process.env.DATABASE_URL as string);
  papr.initialize(client.db());

  await papr.updateSchemas();
})();

export async function disconnect() {
  await client.close();
}

// readyProxy takes an object and returns a proxy that will wait for the
// database to be ready before executing any methods.
export function readyProxy<T extends object>(obj: T) {
  return new Proxy(obj, {
    get(target: any, prop) {
      // not a function, just return the value
      if (typeof target[prop] !== "function") {
        return target[prop];
      }

      // return a function that will wait for the database to be ready
      return async (...args: any[]) => {
        await connection;
        return target[prop](...args);
      };
    },
  }) as T;
}

export default papr;

readyProxy is the important part. It needs to be able to wait for the setup work (await connection)
models/project.ts
Then wrap all models like this:
const Project = readyProxy(papr.model("projects", projectSchema));

export default Project;

Hey @fiws, thanks for the tip man.
I actually already use proxy for database connection, similar to what you did. It was actually one of the first things I tried, and the issue wasn't fixed. This intermittent error is not related to me not connecting before accessing a model, it might be something else I didn't find yet.
This also only happens with papr.

Almost one year later and I'm still having problems with this. But it's even worse now that I have several projects on production, this issue also happens on production on every release. So I need to restart the app until the collection actually initializes.
Is there anything magical I could try? I'm really out of options here. I'm not even sure this issue is specific to Papr.

My suggestion here, it would be to try something like:
import Papr from "papr";
import { MongoClient } from "mongodb";

let _papr: Papr | undefined;
export function getPapr() {
  return _papr;
}

export async function connectToDb(
  databaseUri: string,
  databaseName: string
) {
  if(_papr !== undefined) {
    return;
  }

  _papr = new Papr();
  const client = await MongoClient.connect(databaseUri);
  const database = client.db(databaseName);
  _papr.initialize(database);
}

Then the function connectToDb() must be called when your are initiating Next server.
After that, you can use getPapr() function to create the models using the same instance that was created in the step before.
What do you think?

My suggestion here, it would be to try something like:

import Papr from "papr";

import { MongoClient } from "mongodb";

let _papr: Papr | undefined;

export function getPapr() {

  return _papr;

}

export async function connectToDb(

  databaseUri: string,

  databaseName: string

) {

  if(_papr !== undefined) {

    return;

  }

  _papr = new Papr();

  const client = await MongoClient.connect(databaseUri);

  const database = client.db(databaseName);

  _papr.initialize(database);

}

Then the function connectToDb() must be called when your are initiating Next server.
After that, you can use getPapr() function to create the models using the same instance that was created in the step before.
What do you think?

My current code is almost exactly like that. The only difference is that I a global instead of just a let. And I also don't use a custom server so I can't really run anything anywhere besides inside the API handler.
I also tried try-catching this error and forcing the database to be connected again but it also doesn't work. Somehow, as soon as it happens I can't use Papr anymore (it will always throw the same error).
I think this might have something to do with how Next bundles the project and keeps everything on the same file when deploying the endpoint. But even with this theory, I have no idea how to fix it.

Most probable that's not enough. Try this:
Db file:
import Papr from "papr";
import { MongoClient } from "mongodb";

let _papr: Papr | undefined;

export async function connectToDb(
  databaseUri: string,
  databaseName: string
) {
  if(_papr !== undefined) {
    return _papr;
  }

  _papr = new Papr();
  const client = await MongoClient.connect(databaseUri);
  const database = client.db(databaseName);
  _papr.initialize(database);

  return _papr;
}

Assets file:
import { onlyServer } from "@italodeandra/next/utils/isServer";
import Papr, { schema, types, VALIDATION_ACTIONS, VALIDATION_LEVEL } from "papr";

const assetSchema = onlyServer(() =>
  schema(
    {
      offers: types.array(
        types.object({
          price: types.number({ required: true }),
          name: types.string(),
          createdAt: types.date({ required: true }),
          accepted: types.boolean(),
          repliedAt: types.date(),
        })
      ),
    },
    {
      timestamps: true,
      validationLevel: VALIDATION_LEVEL.OFF,
      validationAction: VALIDATION_ACTIONS.WARN,
    }
  )
);

export function getAsset(papr: Papr) {
  return papr.model("assets", assetSchema)
}

Your API file:
import {getAsset} from "../../../collections/asset/Asset";
import { apiHandlerWrapper } from "@italodeandra/next/api/apiHandlerWrapper";
import connectToDb from "../../../db/db";

async function handler() {
  const papr = await connectToDb();
  const Asset = getAsset(papr); // This should solve the race condition

  return await Asset.find(
    {},
    {
      projection: {
        offers: 1,
      },
      sort: {
        updatedAt: -1,
      },
    }
  );
}

export default apiHandlerWrapper(handler);

This fixed it.


Make spirits randomizable

The sprits that are collected at the end of the first three dungeons should be randomizable

Yes, see #33


staging-next-23.11 iteration 1 - 2023-11-25

workflow docs: https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md#staging
important job: https://hydra.nixos.org/job/nixpkgs/staging-next-23.11/unstable#tabs-constituents
jobsets:
https://hydra.nixos.org/jobset/nixpkgs/staging-next-23.11
https://hydra.nixos.org/jobset/nixos/staging-next-23.11-small
nix-review reports: https://malob.github.io/nix-review-tools-reports/
previous staging-next-23.11 PR: (none)

Significant breakages
(will be edited based on progress)

[ ] libreoffice

[ ] libreoffice fails some test (both platforms, repeatedly, -still only)
https://hydra.nixos.org/build/242584031
/cc maintainer @7c6f434c

That's something expired.  Not introduced by this PR.


Item provides save panel options

This changes the pane.saveItemAs logic to look for a getSaveOptions method and if present those options will be passed into the save dialogue. This add flexibility, giving items more control over how the save dialog works. I need it so that I can include filter types in the save dialogue. But I expect there will be other uses cases going forward.

Awesome, thanks for this :+1:


Strange kernel config merging behaviour

Describe the bug
When including the hardened profile, a set of kernel config options are added to the common config.
This largely works as expected but the module merging seems to behave different from expectations.
New options are merged in properly, existing options are not rewritten.
To Reproduce
Steps to reproduce the behavior:

Import <nixpkgs/nixos/modules/profiles/hardened.nix> into your NixOS config
nixos-rebuild boot
Reboot
zcat /proc/config.gz | grep PAGE_POISONING | head -1 should show CONFIG_PAGE_POISONING=y (correct, new option from hardened config)
zcat /proc/config.gz | grep INET_DIAG | head -1 should show CONFIG_INET_DIAG=n but shows CONFIG_INET_DIAG=m instead. (Wrong, option should be overwritten by hardened config)

You can expand this by setting MODULES=n in common-config.nix, which will fail the build for nix-build -A linux_latest_hardened.configfile because INET_DIAG is not properly set to no.
Expected behavior
Config flags set in hardened config should properly overwrite values from common config.
Additional context

The config is passed into structuredExtraConfig here: https://github.com/NixOS/nixpkgs/blob/d73afe9462a9d61006fff9f1aa6b601129a6d1f7/pkgs/top-level/all-packages.nix#L19531
Common and hardened config get merged here: https://github.com/NixOS/nixpkgs/blob/d73afe9462a9d61006fff9f1aa6b601129a6d1f7/pkgs/os-specific/linux/kernel/generic.nix#L159
The order of the modules to be merged does not seem to matter. Swapping common and hardened configs in the list passed to evalModules does not change anything

Notify maintainers

@teto Added the structured config merging
@joachifm Added the hardened config
@talyz Seems to use the hardened profile
@emilazy Seems to use the hardened profile

Metadata

system: "x86_64-linux"
host os: Linux 5.10.17-hardened1, NixOS, 21.05.git.983f4d6894d (Okapi)
multi-user?: yes
sandbox: yes
version: nix-env (Nix) 2.4pre20201205_a5d85d0
channels(root): "nixos-20.09.3160.5c53c720ff6"
channels(nequi): "home-manager, nixpkgs-21.03pre264299.01291474ea9"
nixpkgs: /home/nequi/.nix-defexpr/channels/nixpkgs

This was already reported. This MR doesn't work but you can follow the linked issues https://github.com/NixOS/nixpkgs/pull/106395
Bottomline is the default merging strategy is not a good idea after all. Someone proposed to remove it altogether. Maybe it's best to disable it by default but it risks to break a few configs so it needs some thought.

@teto Oh! I indeed missed that PR. Thanks. Closing this as a duplicate.


Add Mastodon API

Create an account here
https://botsin.space/
API
https://github.com/andregasser/bigbone

BigBone uses Kotlin and GSON. I might as well just use the JAX-RS client libs and JSON-B...


[move cli] command for analyzing a collection of Move packages

Basic command that can take the output of sui_getObject or sui_multiGetObjects, deserialize the result, and compute some summary stats.
In the future, this + a list of all on-chain packages will be useful as a smoke test for linters, static analyzers, and the bytecode verifier.
Here's the stats for all the packages from mainnet:
Stats {
total_packages: 3879,
total_distinct_packages: 3728,
total_module_bytes: 12856485,
total_modules: 5983,
total_structs: 51452,
total_objects: 6180,
total_hot_potatoes: 197,
total_functions: 16940,
max_modules: 48,
max_structs: 40,
max_functions: 144,
max_struct_fields: 28,
max_function_parameters: 44,
max_dependencies: 18,
max_version: 8
}

Would it be very annoying to suggest splitting this out into its own crate and binary if we want to land it?  I'm struggling to find a way to fit this in to a coherent design for a CLI for builders (which is maybe not the state of the CLI today, but I would like it to get there).  Maybe we also need to create a separate, internal tool/binary for this sort of thing.

I've already thought of a name for our internal binary -- suiss-army-knife ...maybe it's too long.

I'm struggling to find a way to fit this in to a coherent design for a CLI for builders (which is maybe not the state of the CLI today, but I would like it to get there)
Agreed with this goal! Here are my thoughts on how this could fit in (which was not at all explained in the PR description, apologies for that).

I think it makes sense for sui move to support linters, analyzers, and the verifier/linker both on source and on bytecode.  One could argue for including this in the build command, but my mental model for that command is that it's for going from source -> bytecode. If you are instead starting from bytecode, I think it makes sense to do it from a different command (since you might not be able to build from source, or might not want to).
We don't have a way to support the bytecode input "mode" today. I'm far from sure that the way this PR implements this mode (file w/ vector of JSON inputs scraped from multi_get_objects) is the best or only way, but I do think it is something important to have, and that it probably wants to be a different command than build. However, I'm not sure that we want it to be an entirely separate + internal-facing CLI--I think app devs will want to run linters/analyzers and the verifier/linker, and folks doing code analytics or building their own tooling will want this too.
The analyze subcommand as an entrypoint for all things bytecode based (whether that bytecode is hot-off-the-presses from a build, read from an RPC, or in a JSON file) was my thought of a way to support this, but there may well be better ways.

The analyze subcommand as an entrypoint for all things bytecode based (whether that bytecode is hot-off-the-presses from a build, read from an RPC, or in a JSON file) was my thought of a way to support this, but there may well be better ways.
This makes sense to me, and I like the framing of "all things bytecode based" needing its own entrypoint.  My thinking around the separate binary is more related to this point:

I'm far from sure that the way this PR implements this mode (file w/ vector of JSON inputs scraped from multi_get_objects) is the best or only way, but I do think it is something important to have

If we had a separate internal binary, we would be more free to make experimental choices to get the functionality out there without worrying too hard about the organisation and input formats (because we are the customers), and that would give us time to understand what the most approachable interface would be for builders.  So I'm imagining a kind of staging area for these tools, while they have rough edges. Some may eventually be polished and put in sui, others may always remain internal.

If we had a separate internal binary, we would be more free to make experimental choices to get the functionality out there without worrying too hard about the organisation and input formats (because we are the customers), and that would give us time to understand what the most approachable interface would be for builders. So I'm imagining a kind of staging area for these tools, while they have rough edges. Some may eventually be polished and put in sui, others may always remain internal.

Got it + agree with this reasoning. What do you think about putting this into the sui-tool binary?

Yep, very happy to use sui-tool for this.


⚠ Bofrost.de has degraded performance

In 622e7d3, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1457 ms

Resolved: Bofrost.de performance has improved in e18a173.


Ignore CVEs 2021-3533 and 2021-3532 in Ansible 2.9 base image

The following CVEs are causing pipenv check step of image build to
fail:
https://nvd.nist.gov/vuln/detail/CVE-2021-3533
https://nvd.nist.gov/vuln/detail/CVE-2021-3532
Both of these CVEs are against Ansible 2.9. The Ansible team is aware of
them, but they require deliberately insecure configuration from
the user and the Ansible team considers them invalid CVEs. There are no
plans for these to be fixed in the 2.9.z stream and will not cause a
security problem with Ansible-based operators.
Signed-off-by: austin<EMAIL_ADDRESS>

Fixes https://github.com/operator-framework/operator-sdk/issues/5459


Remove "CRM" in standard version

Replace module title "CRM" with "Contacts"
Remove Lead, Client, Supplier segmentation form "Companies"
Remove activities and financial-data tabs from "Companies"

fixed by https://github.com/sulu-io/sulu/pull/923


i have used scanner but still it identifies it as manually

i have displayed messageBox for each BarcodeEntryType and i have scanned a barcode in dynamics POS but it always displaying Manually entered.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: ad5da804-92c1-a7c7-8bb8-b55e5ead8d28
Version Independent ID: 599fb6b7-66bb-7b50-2393-431d68706ca6
Content: BarcodeEntryType Enumeration (Microsoft.Dynamics.Retail.Pos.Contracts.Services)
Content Source: dynamicsax2012-technet/barcodeentrytype-enumeration-microsoft-dynamics-retail-pos-contracts-services.md
Product: dynamicsax-2012
GitHub Login: @Khairunj

Thanks for contacting us. If you need assistance, contact Support using one of the options described here: Support contact options. In particular, you might want to go to the Dynamics 365 community page to get help with your questions from the Dynamics community.


FreeRADIUS Google Dual Factor Authenticator

From @ThomasHabets on October 10, 2014 8:7
Original issue 326 created by achintha85 on 2013-09-26T06:52:57.000Z:
Hi I've been following this article to setup FreeRADIUS Google Dual Factor Authenticator
http://www.supertechguy.com/help/security/freeradius-google-auth
Hours of testing I still can't get it to work. If my /etc/pam.d/radiusd looks like the following it works well with
the following command
radtest test test localhost 18120 testing123

# /etc/pam.d/radiusd - PAM configuration for FreeRADIUS
#

# We fall back to the system default in /etc/pam.d/common-*
#

@include common-auth
@include common-account
@include common-password
@include common-session
--------------------------------------------------------

However if it looks like the following

--------------------------------------------------------
#
# /etc/pam.d/radiusd - PAM configuration for FreeRADIUS
#

# We fall back to the system default in /etc/pam.d/common-*
#

#@include common-auth
#@include common-account
#@include common-password
#@include common-session

auth requisite pam_google_authenticator.so forward_pass
auth required pam_unix.so use_first_pass
--------------------------------------------------------

my log file says the following and auth fails.

--------------------------------------------------------
rad_recv: Access-Request packet from host <IP_ADDRESS> port 43185, id=111, length=56
        User-Name = "test"
        User-Password = "test"
        NAS-IP-Address = <IP_ADDRESS>
        NAS-Port = 18120
Thu Sep 26 16:38:19 2013 : Info: # Executing section authorize from file /etc/freeradius/sites-enabled/default
Thu Sep 26 16:38:19 2013 : Info: +- entering group authorize {...}
Thu Sep 26 16:38:19 2013 : Info: ++[preprocess] returns ok
Thu Sep 26 16:38:19 2013 : Info: ++[chap] returns noop
Thu Sep 26 16:38:19 2013 : Info: ++[mschap] returns noop
Thu Sep 26 16:38:19 2013 : Info: ++[digest] returns noop
Thu Sep 26 16:38:19 2013 : Info: [suffix] No '@' in User-Name = "test", looking up realm NULL
Thu Sep 26 16:38:19 2013 : Info: [suffix] No such realm "NULL"
Thu Sep 26 16:38:19 2013 : Info: ++[suffix] returns noop
Thu Sep 26 16:38:19 2013 : Info: [eap] No EAP-Message, not doing EAP
Thu Sep 26 16:38:19 2013 : Info: ++[eap] returns noop
Thu Sep 26 16:38:19 2013 : Info: [files] users: Matched entry DEFAULT at line 74
Thu Sep 26 16:38:19 2013 : Info: ++[files] returns ok
Thu Sep 26 16:38:19 2013 : Info: ++[expiration] returns noop
Thu Sep 26 16:38:19 2013 : Info: ++[logintime] returns noop
Thu Sep 26 16:38:19 2013 : Info: [pap] WARNING! No "known good" password found for the user.  Authentication may fail because of this.
Thu Sep 26 16:38:19 2013 : Info: ++[pap] returns noop
Thu Sep 26 16:38:19 2013 : Info: Found Auth-Type = PAM
Thu Sep 26 16:38:19 2013 : Info: # Executing group from file /etc/freeradius/sites-enabled/default
Thu Sep 26 16:38:19 2013 : Info: +- entering group authenticate {...}
Thu Sep 26 16:38:19 2013 : Debug: pam_pass: using pamauth string <radiusd> for pam.conf lookup
Thu Sep 26 16:38:19 2013 : Debug: pam_pass: function pam_authenticate FAILED for <test>. Reason: Cannot make/remove an entry for the specified session
Thu Sep 26 16:38:19 2013 : Info: ++[pam] returns reject
Thu Sep 26 16:38:19 2013 : Info: Failed to authenticate the user.
Thu Sep 26 16:38:19 2013 : Info: Using Post-Auth-Type Reject
Thu Sep 26 16:38:19 2013 : Info: # Executing group from file /etc/freeradius/sites-enabled/default
Thu Sep 26 16:38:19 2013 : Info: +- entering group REJECT {...}
Thu Sep 26 16:38:19 2013 : Info: [attr_filter.access_reject]    expand: %{User-Name} -> test
Thu Sep 26 16:38:19 2013 : Debug:  attr_filter: Matched entry DEFAULT at line 11
Thu Sep 26 16:38:19 2013 : Info: ++[attr_filter.access_reject] returns updated
Thu Sep 26 16:38:19 2013 : Info: Delaying reject of request 0 for 1 seconds
Thu Sep 26 16:38:19 2013 : Debug: Going to the next request
Thu Sep 26 16:38:19 2013 : Debug: Waking up in 0.9 seconds.
Thu Sep 26 16:38:20 2013 : Info: Sending delayed reject for request 0
Sending Access-Reject of id 111 to <IP_ADDRESS> port 43185
Thu Sep 26 16:38:20 2013 : Debug: Waking up in 4.9 seconds.
Thu Sep 26 16:38:25 2013 : Info: Cleaning up request 0 ID 111 with timestamp +3
Thu Sep 26 16:38:25 2013 : Info: Ready to process requests.
--------------------------------------------------------

Can you please tell me what's the issue here?

I'm using Ubuntu latest

Copied from original issue: google/google-authenticator#325

From @tecman77 on March 6, 2015 19:32
I have similar problem with Freeradius and pam_google_authenticator module. I tried  to deploy this on Redhat 6, 7, Fedora 21 without success. Time on server is configured correctly and I am able to log in with two-factor-auth via SSH on the very same server, however FreeRadius doesn't seem to work on these systems. The only system I am able to make it work is Ubuntu so far without any problems. I followed same setup guide as OP.
My pam_google_authenticator was compiled instead of installed from rpm but freeradius - from rpm.
Redhat 7 comes with Freeradius 3.x by default, Ubuntu 2.1. I also compiled freeradius from sources on both systems, and that didn't make any difference. Ubuntu works, Centos/Redhat/Fedora - doesn't with FreeRadius, but SSHD works fine with dual factor authentication.
There is not enough debug options that I could use to troubleshoot it further myself.

From @pantsman0 on October 3, 2015 11:50
I have set this up on RHEL6&7 and CentOS6&7, and I've had it work using the steps in the guide above.
The problem seems to be this line:
Thu Sep 26 16:38:19 2013 : Debug: pam_pass: function pam_authenticate FAILED for <test>. Reason: Cannot make/remove an entry for the specified session
This happens to me when I forget to change the running user/group from radiusd to running as root. You need to run it as root, or move the secrets out of user home directories.
If you continue to have issues, comment here and I will respond when I can.


update to upstream

copied the stuff out of rust-lang/rust src/tools/rustbook into the dir and committed it. this fixes the question_mark thing.

Thanks!


Generate float from JSON Schema

TLDR
Outlines currently fails to generate floats from a JSON schema. This PR fixes that.
The Problem
Here is the smallest working example of where this issue arises.
Import these packages
# !pip install outlines torch transformers datasets accelerate pyairports pycountry
import json
from outlines import models, generate
from transformers import AutoModelForCausalLM, AutoTokenizer

and consider a function that takes in floats
def add(x: float, y: float) -> float:
    """Add two floats.

    Args:
        x (float): The first float.
        y (float): The second float.
    """
    assert isinstance(x, float)
    assert isinstance(y, float)
    return x + y

and here is the same function in a json schema
schema_json = {
    'title': 'add',
    'type': 'object',
    'description': 'Add two floats.',
    'properties': {'x': {'type': 'float', 'description': 'The first float.'},
                    'y': {'type': 'float', 'description': 'The second float.'}},
    'required': ['x', 'y'],
    }

Imagine we also have a model, let's say mistral-7b-instruct-v0.2
model_id = "mistralai/Mistral-7B-Instruct-v0.2"
device = "cuda"
llm = AutoModelForCausalLM.from_pretrained(model_id, token=<YOUR-HF-TOKEN>, trust_remote_code=True).to(device)
tokenizer = AutoTokenizer.from_pretrained(model_id, token=<YOUR-HF-TOKEN>, trust_remote_code=True)
model = models.Transformers(llm, tokenizer)

Now we want to generate arguments for add, and can use the json structured generator to do so
schema_str = json.dumps(schema_json)
generator = generate.json(model, schema_str)

function_args = generator("I am traveling 3 m/s and then travel 2 m/s faster. Can you call the function add to find out my new speed?", max_tokens=30)
print("Input:", function_args)

result = add(**function_args)
print("Output:", result)

This results in the error
---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
Cell In[36], line 1
----> 1 generator = generate.json(model, schema_str)
      3 function_args = generator(\"I am traveling 3 m/s and then travel 2 m/s faster. Can you call the function add to find out my new speed?\", max_tokens=30)
      4 print(function_args)

File /usr/lib/python3.11/functools.py:909, in singledispatch.<locals>.wrapper(*args, **kw)
    905 if not args:
    906     raise TypeError(f'{funcname} requires at least '
    907                     '1 positional argument')
--> 909 return dispatch(args[0].__class__)(*args, **kw)

File /mnt/workdisk/eitan/venvs/outlines-bug/lib/python3.11/site-packages/outlines/generate/json.py:58, in json(model, schema_object, sampler, whitespace_pattern)
     56 elif isinstance(schema_object, str):
     57     schema = schema_object
---> 58     regex_str = build_regex_from_schema(schema, whitespace_pattern)
     59     generator = regex(model, regex_str, sampler)
     60     generator.format_sequence = lambda x: pyjson.loads(x)

File /mnt/workdisk/eitan/venvs/outlines-bug/lib/python3.11/site-packages/outlines/fsm/json_schema.py:83, in build_regex_from_schema(schema, whitespace_pattern)
     80 resolver = registry.resolver()
     82 content = schema.contents
---> 83 return to_regex(resolver, content, whitespace_pattern)

File /mnt/workdisk/eitan/venvs/outlines-bug/lib/python3.11/site-packages/outlines/fsm/json_schema.py:142, in to_regex(resolver, instance, whitespace_pattern)
    140 for i, (name, value) in enumerate(properties.items()):
    141     subregex = f'{whitespace_pattern}\"{re.escape(name)}\"{whitespace_pattern}:{whitespace_pattern}'
--> 142     subregex += to_regex(resolver, value, whitespace_pattern)
    143     if i < last_required_pos:
    144         subregex = f\"{subregex}{whitespace_pattern},\"

File /mnt/workdisk/eitan/venvs/outlines-bug/lib/python3.11/site-packages/outlines/fsm/json_schema.py:356, in to_regex(resolver, instance, whitespace_pattern)
    349         regexes = [
    350             to_regex(resolver, {\"type\": t}, whitespace_pattern)
    351             for t in instance_type
    352             if t != \"object\"
    353         ]
    354         return rf\"({'|'.join(regexes)})\"
--> 356 raise NotImplementedError(
    357     f\"\"\"Could not translate the instance {instance} to a
    358 regular expression. Make sure it is valid to the JSON Schema specification. If
    359 it is, please open an issue on the Outlines repository\"\"\"
    360 )

NotImplementedError: Could not translate the instance {'type': 'float', 'description': 'The first float.'} to a
    regular expression. Make sure it is valid to the JSON Schema specification. If
    it is, please open an issue on the Outlines repository"
}

where the most important part is the error raised in the to_regex function
NotImplementedError: Could not translate the instance {'type': 'float', 'description': 'The first float.'} to a
    regular expression. Make sure it is valid to the JSON Schema specification. If
    it is, please open an issue on the Outlines repository

This is very strange. From some reason outlines cannot turn {'type': 'float', 'description': 'The first float.'} into regex. This seems very simple. What is going on?

Instead of passing a string into generate.json (i.e. generate.json(model, schema_str)), I tried passing the function directly into generate.json
generator = generate.json(model, add)

function_args = generator("I am traveling 3 m/s and then travel 2 m/s faster. Can you call the function add to find out my new speed?", max_tokens=30)
print("Input:", function_args)

result = add(**function_args)
print("Output:", result)

and I get
Input: {'x': 6, 'y': 0}
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
Cell In[6], line 6
      3 function_args = generator(\"I am traveling 3 m/s and then travel 2 m/s faster. Can you call the function add to find out my new speed?\", max_tokens=30)
      4 print(\"Input:\", function_args)
----> 6 result = add(**function_args)
      7 print(\"Output:\", result)

Cell In[3], line 10, in add(x, y)
      3 def add(x: float, y: float) -> float:
      4     \"\"\"Add two floats.
      5
      6     Args:
      7         x (float): The first float.
      8         y (float): The second float.
      9     \"\"\"
---> 10     assert isinstance(x, float)
     11     assert isinstance(y, float)
     12     return x + y

AssertionError: "
}

Observe that running generator = generate.json(model, add) here gives us a different error than when we ran generator = generate.json(model, schema_str).
When we ran generator = generate.json(model, schema_str) there was an issue creating the generator. With generator = generate.json(model, add) we successfully create generator and get that the function arguments generated by the model are {'x': 6, 'y': 0}. However, the model generated integers (i.e. 6, 0) and not floats (i.e. 6.0, 0.0).
What is going on here?
The solution
There are a couple of issues here:

In json_schema.py there was no FLOAT type. This is why we got the error NotImplementedError: Could not translate the instance {'type': 'float', 'description': 'The first float.'} to a regular expression. I added the FLOAT type to that file. This fixed the first issue where the generator cannot convert schema_str to regex in generator = generate.json(model, schema_str).
Although we don't have the FLOAT type in json_schema.py, we do have the FLOAT type in types.py:

INTEGER = r"[+-]?(0|[1-9][0-9]*)"
FLOAT = rf"{INTEGER}(\.[0-9]+)?([eE][+-][0-9]+)?"

The ? in (\.[0-9]+)? means that for a float it is optional to include a decimal with some numbers after it. This means we can have floats without a decimal afterwards. Wait... isn't that just an integer! This regex pattern is incorrect. So I changed this regex pattern to
FLOAT = rf"[+-]?[0-9]*\.[0-9]+([eE][-+]?[0-9]+)?"

which requires that we have a decimal in the float.
3. I also noticed that we have two different definitions of INTEGER. In json_schema.py we have INTEGER = r"(-)?(0|[1-9][0-9]*)" and intypes.py we have INTEGER = r"[+-]?(0|[1-9][0-9]*)". I changed the definition in json_schema.py to be the same as that in types.py because INTEGER in types.py allows both + and - in front of the number and the definition in json_schema.py does not allow this.
TO DO: More generally, I think we should store these types in one single location. I'm thinking of importing all these definitions from types.py into json_schema.py, that way we won't have diverging definitions again.

But what about our second issue, that generator = generate.json(model, add) generated integers and not floats? Recall that when the schema_object is of type callable, the following lines are called:

schema = pyjson.dumps(get_schema_from_signature(schema_object))
regex_str = build_regex_from_schema(schema, whitespace_pattern)

The function get_schema_from_signature calls Pydantic's model.model_json_schema() function to convert a Pydantic model to a JSON schema. For some reason, this model_json_schema() casts objects of type floats to objects of type number. I don't know why this occurs. See this Github issue for more info.
So calling schema = pyjson.dumps(get_schema_from_signature(schema_object)) takes all floats in our function and casts them to type number. Then when we call build_regex_from_schema, we take any objects of type number and turn it to the regex pattern
NUMBER = rf"({INTEGER})(\.[0-9]+)?([eE][+-][0-9]+)?"

where NUMBER represents any integer or float. Focusing on (\.[0-9]+)?, notice that the ? means that it is optional to add the decimal with a number after it. All of our floats are turned to type number whose regex forces that number to be either an int or a float.
This is why calling generator = generate.json(model, add) with floats in add ends up with us generating ints. I don't know the best way to fix this and so I made a PR to Pydantic about this.
To Do

Should we delete the types in json_schema.py and instead import the types from types.py?
How do we generate turn a function into a JSON schema without casting float to number?

Any thoughts would be appreciated.

I know this is a very long/complex issue but would love help fixing this as soon as possible. Thanks!

As in #888 please refer to the JSON definition and JSON Schema specification.

Hi @rlouf!
I'm planning on breaking this PR up into smaller PRs as there are many different changes I made here at once. (The first of these is the PR #888.)
I will look through the JSON schema like you suggested. In the meantime my code breaks anytime I have a float in my JSON schema. I'll create a PR for just this issue now because it is the most pressing one.

After reading this JSON Schema Specification, I see that JSON does not actually support float; it only supports int or number.
Man, that is frustrating.
So the JSON schema I provided in my example
schema_json = {
    'title': 'add',
    'type': 'object',
    'description': 'Add two floats.',
    'properties': {'x': {'type': 'float', 'description': 'The first float.'},
                    'y': {'type': 'float', 'description': 'The second float.'}},
    'required': ['x', 'y'],
    }

would never actually be generated in practice as there is no type float.
Additionally, the fact that schema = pyjson.dumps(get_schema_from_signature(schema_object)) takes all floats in our function and casts them to type number is actually behavior that now makes sense.
This clears a lot of things up, thank you!

Is there a way to use generate.json() to get the model to output floats in a JSON schema? This float would be consistent with the Python definition of float, i.e. it must have a decimal point and one or more numbers after the decimal point, i.e. r"[+-]?[0-9]*\.[0-9]+([eE][-+]?[0-9]+)?"?
This definition of float is different from the current definition of number, which optionally has a decimal point?
Thanks!

Is there a way to use generate.json() to get the model to output floats in a JSON schema? This float would be consistent with the Python definition of float, i.e. it must have a decimal point and one or more numbers after the decimal point, i.e. r"[+-]?[0-9]*\.[0-9]+([eE][-+]?[0-9]+)?"?
This definition of float is different from the current definition of number, which optionally has a decimal point?
Thanks!

We could add a outlines.types.Float custom type. Thinking about it we may take some liberties wrt the JSON spec, but should open an issue to discuss it prior to opening a PR.

Closing as discussed offline. Generating Python-compatible strings may lead to non-parseable JSON.


Output change by rescue_read refactor

Commit 13bdc5d introduced a rescue_read function that is used for the cases that either read 1 or read 2 need to be rescued.
The refactoring changes the output in some cases (but not the accuracy) for the following reason. Previously, there was this function call for both cases (rescue read 1 or rescue read 2): get_best_scoring_pair(aln_scores1, aln_scores2, ...) In both cases, read 1 was the first argument and read 2 the second. With the refactoring, the role of read 1 and 2 is reversed in one of the cases. If there are multiple best scoring pairs, it appears that the sort then gives different results:
https://github.com/ksahlin/StrobeAlign/blob/e758839978c3df013bccae22ec70b695d7062c49/src/aln.cpp#L1477
The function is otherwise symmetric so that it doesn’t matter which read is 1 and which is 2, so I would tend to leave this as it is (also because accuracy does not change, only how ties are broken), but I wanted to mention it.

Ok, since @ksahlin gave me a :+1:, I’ll close this without taking any further action.

Yeah I am not sure I have a good suggestion. Seems okay to ignore for now though?
I have just set off an evaluation of latest master (commit 464a852), let's see. I am currently only benchmarking index-and-align mode. I will then include also the 'first index, then alignment' option run (with this https://github.com/ksahlin/StrobeAlign/pull/134), but looking at the two separate modes and runtime difference is not the priority right now.


[Wait 4 #385] Refactor neuralnet::init

Changes proposed in this PR:

Add determinancy and check to
Delete cost / bn_follow from layer
Enable negative test for modelfile
fix IniSection::setEntry is wrongly using key for an erase behavior
Delete VERIFY_SET_DIMENSION()

Resolves #316
See also #374
Self evaluation:

Build test: [X]Passed [ ]Failed [ ]Skipped
Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee<EMAIL_ADDRESS>

:memo: TAOS-CI Version: 1.4.20191203. Thank you for submitting PR #384. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://nnsuite.mooo.com/.

:octocat: cibot: @zhoonit, One of the module of prebuild group is failed. If you want to get a hint to fix this issue, please go to https://github.com/nnstreamer/nntrainer/wiki/.

:octocat: cibot: @zhoonit, A builder checker could not be completed because one of the checkers is not completed. In order to find out a reason, please go to http://nnsuite.mooo.com/nntrainer/ci/repo-workers/pr-checker/384-202007292106480.86871004104614-4804013b72f5df76820a6ae8de82a8204b63954c/.

:octocat: cibot: @zhoonit, One of the module of prebuild group is failed. If you want to get a hint to fix this issue, please go to https://github.com/nnstreamer/nntrainer/wiki/.

:octocat: cibot: @zhoonit, One of the module of prebuild group is failed. If you want to get a hint to fix this issue, please go to https://github.com/nnstreamer/nntrainer/wiki/.

:octocat: cibot: @zhoonit, A builder checker could not be completed because one of the checkers is not completed. In order to find out a reason, please go to http://nnsuite.mooo.com/nntrainer/ci/repo-workers/pr-checker/384-202007301124180.59592795372009-710f47e67215e85ba5216474d25971a5e70e02e7/.

:octocat: cibot: @zhoonit, A builder checker could not be completed because one of the checkers is not completed. In order to find out a reason, please go to http://nnsuite.mooo.com/nntrainer/ci/repo-workers/pr-checker/384-202007301133290.27506804466248-e68155df05eeac9307fa19f7e003cf8d97d34418/.


Loading class that is the parent of the app container

Right now, the ._dash-loading-indicator class is a cousin of the app container. If it was the parent, then you could customize UI in your app.layout that would only appear while a callback is being executed:
app.layout = html.Div(className='display-on-loading', children='Loading')

.display-on-loading {
    display: none;
}
._dash-loading-indicator  .display-on-loading {
    display: block;
}

I'm assuming this is closed by the loading states API https://github.com/plotly/dash-renderer/pull/93 https://community.plot.ly/t/loading-states-and-loading-component/19650 - if there's anything still relevant here please reopen in the dash repo.


Add an example on how to integrate Matplotlib

What would that feature address
Many Python programmers are using Matplotlib as a way to create charts.
These are static, and we want to let users in charge of defining their charts entirely.
Description of the ideal solution
The best way to do this seems to be using mpld3 and the plugins module.
An example would show how to create a Matplotlib chart, and how to integrate it in a Taipy page (certainly using the figure property of the chart control).
Acceptance Criteria

[ ] The example is part of the documentation set.

Hi @FabienLelaquais @jrobinAV
Can you please assign me this issue under Hacktoberfest and Devfest ai?

Thank you @jrobinAV sir, will get started on it now.

Hii @jrobinAV can you assign this issue to me?

Sir have raised the PR, please check

Hi There,
If this is still waiting for a contributor, can you assign this to me as part of Hacktobeberfest.? I can submit a matplotlib example or two using the part control with a content provider.
I have two examples:
(1) Displaying an astronomy image (download from the internet) in FITS format, along with metadata displayed in a table using the AstroPy library that professional astronomers use.
(2) I also have an example that uses matplotlib along with 'part' and a 'content provider' to display a latex equation, using matplotlib's internal mathtex implementation (a subset of tex). And if you want something really cool, I created one that will generate mathml, and use an SVG trick to fallback to a matplotlib generated png image of the equation on browsers that do not support mathml. (I'm also working on creating a content provider that will use the katex library clientside to display the equations, but it is still a work in progress.)
Cameron :-)
(Deep Sky Wonder)

Hi @jrobinAV If this issue still needs contributions, I'd like to give it a good try

Hi @AbisoyeOnanuga, Please feel free to work on it and propose a PR.
You can also review the existing PRs and eventually collaborate with @SGCODEX to benefit from each other's experience.

@jrobinAV @FredLL-Avaiga @FabienLelaquais , if #2179 as proposed by @AbisoyeOnanuga solves the markdown example directly without using the save image feature. Then we can go ahead with that for the markdown example and my example for the tgb can be used as mentioned in earlier parts of #1930. Please let me know the changes I should do in my PR. Thank you.

@jrobinAV @FredLL-Avaiga @FabienLelaquais after updating to Taipy 4.0.0 my solution https://github.com/Avaiga/taipy/pull/2179 reverts to using saved image feature. Either markdown example is okay and your example for the tgb can be used from https://github.com/Avaiga/taipy/pull/1930.
I used the same folder structure as @SGCODEX if that helps


watchtower unable to detect image

Hello,
Watchtower is unable to find image name of most of my containers.
It gives this output:
time="2017-07-18T22:46:42Z" level=debug msg="Pulling sha256:72323629575287eb57b46b1b1eec223af3f0fa0b660c43cc156a3cf5c99b9e0e for /muximux"
time="2017-07-18T22:46:42Z" level=debug msg="No credentials for sha256 in /config.json"
time="2017-07-18T22:46:42Z" level=debug msg="No authentication credentials found for sha256:72323629575287eb57b46b1b1eec223af3f0fa0b660c43cc156a3cf5c99b9e0e"
time="2017-07-18T22:46:44Z" level=debug msg="Error pulling image sha256:72323629575287eb57b46b1b1eec223af3f0fa0b660c43cc156a3cf5c99b9e0e, Error response from daemon: repository sha256 not found: does not exist or no pull access"

Some of them is working and give this output:
time="2017-07-18T22:46:37Z" level=debug msg="Pulling v2tec/watchtower:latest for /watchtower"
time="2017-07-18T22:46:37Z" level=debug msg="No credentials for v2tec in /config.json"
time="2017-07-18T22:46:37Z" level=debug msg="No authentication credentials found for v2tec/watchtower:latest"
time="2017-07-18T22:46:38Z" level=debug msg="No new images found for /watchtower"
time="2017-07-18T22:46:38Z" level=debug msg="Pulling linuxserver/openvpn-as:latest for /openvpn-as"
time="2017-07-18T22:46:38Z" level=debug msg="No credentials for linuxserver in /config.json"
time="2017-07-18T22:46:38Z" level=debug msg="No authentication credentials found for linuxserver/openvpn-as:latest"
time="2017-07-18T22:46:40Z" level=debug msg="No new images found for /openvpn-as"

Seems like for the first one you have a container running a specific image sha256:72323629575287eb57b46b1b1eec223af3f0fa0b660c43cc156a3cf5c99b9e0e and not an image with a repository and name like linuxserver/openvpn-as:latest
How do you have created the container muximux? Could you please post your docker run command?

Closing due to lack of response. If this is still an issue, feel free to let me know and I'll take a look.

Hi! It's happening to me to, I think.
I'm using a docker stack with this:
web_my_service:
    image: xpto.com/my_service:latest
    labels:
      com.centurylinklabs.watchtower.enable: "true"

and then I run watchtower with this:
sudo docker run -d \
  --name watchtower \
  -v /root/.docker/config.json:/config.json \
  -v /var/run/docker.sock:/var/run/docker.sock \
  v2tec/watchtower  --debug --label-enable --interval 30

Here's the log:
time="2019-06-05T11:56:47Z" level=debug msg="Pulling xpto.com/my-service:latest@sha256:f24be6b60cdf4d99b8a57ffcd888eb4807c8cdbcbfbeb2b682733c6538fbab19 for /stack_name__web_my_service.1.lltio3h9tq3erj1osjblxo34w"
time="2019-06-05T11:56:47Z" level=debug msg="Loaded auth credentials {xpto registry   xpto.com  } from /config.json"
time="2019-06-05T11:56:47Z" level=debug msg="No new images found for /stack_name__web_my_service.1.lltio3h9tq3erj1osjblxo34w"

And I know there's a new image because if I pull it manually, I get a new one. Watchtower seems to be pointing to a specific "sha" with the "@" symbol and not recognizing the new versions by image id.
Am I doing something wrong?
Thanks!

Watchtower currently isn't compatible with docker swarm and stacks. Docker swarm pins the version of the image used to the version which is available during the creation of the service and the tasks (containers) of the service will be created with the pinned version. This is especially needed if you have several nodes and all these nodes should have the same version of the image running. Therefore watchtower tries to get the newest image of the pinned version, which is always the same version.
So as long as watchtower doesn't support docker swarm (docker services), it will not be possible to update services of stacks deployed on swarm.

There is already an issue for that though so I don’t think this one needs to be reopened.


[Bug] inlineSyncPlugin triggers markdownUpdated event

Initial checklist

[X] I agree to follow the code of conduct
[X] I searched issues and discussions and couldn’t find anything (or linked relevant results below)

Affected packages and versions
7.1.0
Link to runnable example
No response
Steps to reproduce

use listener & commonmark plugins
listen to markdownUpdated event
paste an image into editor
markdownUpdated event was triggered extra twice.

Expected behavior
inlineSyncPlugin doesn't trigger any event since it's an internal mechanism.
Actual behavior
markdownUpdated event was triggered extra twice.
Runtime
Electron
OS
No response
Build and bundle tools
No response

I found another problem caused by inlineSyncPlugin maybe:

define and use a $nodeSchema(multimedia node)
define and use a corresponding $view(multimedia node view)
use a custom uploader for @milkdown/plugin-upload:

  async uploader(files, schema) {
    // some async business log to upload files and get therir id on Server
    // ...(omit)
    // ...
    // create and return multimedia node here, with `mimeType` attr
    // ** make multimedia node view constructor invoked with `mimeType` attr**, excepted
    return updatedFiles.map((file) => {
      const node = multimediaNode.createAndFill({
        src: `${fileProtocol}:///${file.id}`,
        alt: file.name,
        mimeType: file.mimeType,
      });

      if (!node) {
        throw new Error('create node failed');
      }

      return node;
    });
  },

node view constructor invoked again unexpectedly, losing mimeType attr. Here is call stack:

Is inlineSyncPlugin breaking ProseMirror's diff algorithm or something, which cause the multimedia is re-created by markdown text? (![](file.id))

It's caused by your node schema doesn't serialize to markdown correctly. I still think instead of creating a new node. You should extend the image schema if you just want to change the behavior of toDOM and parseDOM, like what I did in task list item.

Ok, choosing the proper plan for features is case-depending, but how about this:
https://github.com/Milkdown/milkdown/blob/74409b93b692df3bf881b2e4644711e3d954b409/packages/preset-commonmark/src/plugin/inline-sync-plugin/context.ts#L97
Should we copy all attrs of all descendants here, instead of the most outer node?

It's by design because after the syncing happens, we don't know what exactly happened inside the paragraph or heading node (based on the preset). So we just let the markdown parser take over the job and give us the right result.


feat: support arbitrary length contracts

BREAKING CHANGE: the various Contract2, Contract3, ... types have all been replaced with a single Contract type that takes a tuple of Runtype validators.

Coverage increased (+0.003%) to 99.194% when pulling<PHONE_NUMBER>8a0f15e9b0e9b74f251c7d8192c31f on yuhr:feat/arbitrary-length-contracts into<PHONE_NUMBER>79fba77a709050b4d0b393fb1dadc6 on pelotom:master.


Any base image other than the cached ones fail on docker for windows

Description
If you base a container on a base image other than the cached on, for example mcr.microsoft.com/windows/servercore:20H2.
docker fails with the following error:

no matching manifest for windows/amd64 10.0.17763 in the manifest list entries

Area for Triage:

Question, Bug, or Feature?:

Bug
Virtual environments affected

[ ] Ubuntu 16.04
[ ] Ubuntu 18.04
[ ] Ubuntu 20.04
[ ] macOS 10.15
[ ] macOS 11
[ ] Windows Server 2016 R2
[x] Windows Server 2019

Image version

Image version where you are experiencing the issue.
Microsoft Windows Server 2019
10.0.17763
Datacenter
Environment: windows-2019
Version: 20210509.1
Included Software: https://github.com/actions/virtual-environments/blob/win19/20210509.1/images/win/Windows2019-Readme.md
Image Release: https://github.com/actions/virtual-environments/releases/tag/win19%2F20210509.1
Expected behavior
The proper base image to be pulled, or a documented reason why it will not
Actual behavior
Just errors out with:

no matching manifest for windows/amd64 10.0.17763 in the manifest list entries

Repro steps
A description with steps to reproduce the issue. If your have a public example or repo to share,
please provide the link.
build the following image on the virtual environment:
FROM mcr.microsoft.com/windows/servercore:20H2

RUN echo "hello"

Hello @jelmansouri-legion,
Could you please try to use FROM mcr.microsoft.com/windows/servercore:20H2-amd64 image?

I'm guessing the error above is what should have been displayed in the first place?

@jelmansouri-legion I'm going to close the issue. In case of any questions feel free to contact us.

Thanks @Darleev !


Dependency Analysis

I am planning to perform dynamic code analysis after succesful containerization, which information I can store in a knowledgebase.
This information will help us with static analysis of new projects

@skoulouzis Lets discuss the results on friday

Section 4.3 discusses the findings of the AdvancedHiddenDependencies component (with the example of pls/glmnet described in this ticket)
Master_Thesis___Final (1).pdf


Support for migration logic in installation

Description
In the current installer, we usually have migration logic written in a job being triggered by helm hooks. That approach we want to avoid in future:

to not rely on helm features (a simple "helm template - kubectl apply" should work as well)
to not provide an option for writing dirty shell scripts
to force people taking care about a smart stimulous. Do not just rely on pre-upgrade hook trigger and execute long-running job always but really think about when to execute dependent on a cluster state. If the cluster is already migrated the code should perform a minimal number of operations and should not consume time and resources.

Goal: Provide a way in the new installer to cover migration logic which usually should be executed only at max once in the life of a cluster. The new way should be golang based so that you can have it tested
Criterias:

there is a well-defined place to put my migration logic in golang source code
There is a well-defined interface to be implemented and to register my struct in order to get trigger on any install/upgrade
The interface provides me access to the current kubernetes client instance and the components which are about to be processed
the logic will be triggered on any upgrade/install and the logic needs to decide if it should be executed or not (check component list and do lookups in the cluster to determine state, do not make it dependent on version but actual state)
a Pre- and Post execution option probably is required
There is a guide on how to mark your logic with a deprecation note to know when it should be removed
the migration execution time should be logged to easily spot the code that consumes too much time (after the migration is done the next run should execute in no time).

Reasons
Migrations are simply required, see for example attached issue. Changing the size of a PV for a statefulSet requires custom actions.

Attachments

https://github.com/kyma-project/kyma/pull/11132

A comment on the upgrade scope:
We have two kinds of possible migration actions:

generic, component-independent. These should be applied regardless of any components being upgraded
specific for a single component only.

Generic migrations should be applied before upgrade starts.
Component-specifc migrations may be applied just before the component is upgraded.
This is important if we want to support a model where we just upgrade a set of Components (or a single component only).
If we assume we always upgrade entire Kyma, then all migrations may be applied together before upgrade starts.

A comment about cleanup/uninstall
If we want a clean "uninstall", we should be also able to "undo" any cluster-wide changes introduced by the migration logic.
Perhaps we should consider a "uninstall" lifecycle phase  in our interface. So that the developer has a chance to remove/unset anything Kyma-specific from the cluster.
But then, how easy it is to do it over multiple kyma-cli versions?
If we install Kyma using version 1.30, then upgrade through 1.31, 1.32 ... etc. up to 1.40, will kyma-cli v.1.40 be able to "undo" things from 1.31 ?

Documented Design for PoC (Draft):
https://github.com/kyma-project/community/blob/main/internal/proposals/migration-hooks/migration-hooks-proposal.md

PoC Implementation can be found here. This then can/should also be used as the start for the implementation.
The results, etc. have been documented in this PR


[AT-752] Integrate sonarcloud

Jira Ticket
Integrated Sonarcloud.
NOTE: @chrieke @andher1802  I did not bump version. Don't think that's required.
Items:

[ ] Ran test & live-tests
[ ] Implemented (new) unit tests
[ ] Removed credentials
[ ] Removed argument pointing to staging
[ ] Updated documentation

For release:

[ ] Bumped version
[ ] Added changelog
[ ] Updated announcement banner

Can only review surface level by comparing with the docs, but seems to work as expected. Thanks a lot @seedlit !


Replace image with same file name does not work in dev mode

Now I need to restart vite and clear the browser cache to get the image to change successfully.
Maybe we should add mtimeMs stat info in the file id here:
https://github.com/JonasKruckenberg/imagetools/blob/main/packages/core/src/util.ts#L9

A workaround is to attach a custom config (such as file hash, or time as you suggested). Example:
// vite.config.ts
imagetools({
  defaultDirectives(url) {
    const buffer = readFileSync(url.pathname);
    const hash = createHash("sha1").update(buffer).digest("hex");

    return new URLSearchParams({ hash });
  },
}),

You might also need to disable sharp caching, since the file path won't change.
import sharp from "sharp";
sharp.cache(false);


Integrated code lifecycle: Improve Docker exception handling and fix race conditions

Checklist
General

[x] I tested all changes and their related features with all corresponding user types on a test server.
[x] Language: I followed the guidelines for inclusive, diversity-sensitive, and appreciative language.
[x] I chose a title conforming to the naming conventions for pull requests.

Server

[x] Important: I implemented the changes with a very good performance and prevented too many (unnecessary) database calls.
[x] I strictly followed the server coding and design guidelines.
[x] I documented the Java code using JavaDoc style.

Changes affecting Programming Exercises

[x] High priority: I tested all changes and their related features with all corresponding user types on a test server configured with the integrated lifecycle setup (LocalVC and LocalCI).

Motivation and Context

We encountered some issues where exceptions that occured during Docker image pulls where not handled causing the build job to get stuck.
Description

I moved the image pull logic to the build job execution and increased the timeout so there is enough time for the image to be pulled. Exceptions are now handled gracefully.
I also made the build job cancellation more robust and a locking mechanism to the build agent update logic to ensure consistency
Steps for Testing

Prerequisites:

1 Admin

Create a programming exercise with the following Docker image: gitlab.lrz.de:5005/gbs-cm/gbs-tester/gbs:0.0.5
Check the build queue and build agent view. The build jobs for template and solution should not get stuck in the Running  Jobs view and should show up in the recent jobs view of the build agents as ERROR

Testserver States

[!NOTE]
These badges show the state of the test servers.
Green = Currently available, Red = Currently locked

Review Progress

Performance Review

[x] I (as a reviewer) confirm that the client changes (in particular related to REST calls and UI responsiveness) are implemented with a very good performance
[x] I (as a reviewer) confirm that the server changes (in particular related to database calls) are implemented with a very good performance

Code Review

[x] Code Review 1
[x] Code Review 2

Manual Tests

[x] Test 1
[x] Test 2

Summary by CodeRabbit

Refactor

Removed the queueSizeLimit field for simplification.
Enhanced Docker image handling before build job execution.
Introduced synchronization and locking mechanisms for improved build job management and cancellation control.
Improved exception handling in Docker image pulling process.
Added thread safety to build agent information updates.

demonstrated in testing session, works as expected


Change chunking/splitting method from SentenceWindowNodeParser to SentenceSplitter or SemanticSplitterNodeParser

I am getting very bad responses from the LLM when chatting with the file I uploaded. I am using Qdrant as a vector DB and I checked the contents in the database and realized that all the chunks are only one sentence long.
This is because by default privategpt uses SentenceWindowNodeParser to split the text in ingest_service.py
node_parser = SentenceWindowNodeParser.from_defaults()
instead of the default I tried implementing SentenceSplitter like this:
node_parser = SentenceSplitter.from_defaults(chunk_size=1024, chunk_overlap=200)
I also tried implementing SemanticSplitterNodeParser like this:
        ollama_embedding = OllamaEmbedding(
            model_name="nomic-embed-text:latest",
            base_url="http://localhost:11434",
            ollama_additional_kwargs={"mirostat": 0},
        )

        node_parser = SemanticSplitterNodeParser(buffer_size=5, embed_model=ollama_embedding)

Both implementations work and the embedding happens without errors, but when I check the database the file is still split by sentences instead of 1024 characters in case of SentenceSplitter or by meaning in case of SemanticSplitterNodeParser.
Should I make some other changes and what am I missing? Why is splitting by sentence the default if it's so ineffective?
I run privategpt with:
PGPT_PROFILES=vllm make run
Thanks in advance for any help!

Try for example
node_parser = SentenceWindowNodeParser.from_defaults(window_size=20)

No need to pass Sentence Splitter dynamically as it is by default present in Service Context Class definition.
Below is the configuration in Service Context.
node_parser = (
text_splitter  # text splitter extends node parser
or node_parser
or _get_default_node_parser(
chunk_size=chunk_size or DEFAULT_CHUNK_SIZE,
chunk_overlap=chunk_overlap or SENTENCE_CHUNK_OVERLAP,
callback_manager=callback_manager,
)
)
If you don't pass any splitting method in the transformations , it will go to _get_default_node_parser which is using SentenceSplitter. Follow below steps:
Go to ingest_service.py and in transformations remove node parser
transformations=[node_parser,embedding_component.embedding_model]

Currently, by default, we use the sentence division as you have said. We have chosen this way because several articles have commented that it is the simplest but one of more effective way to do it. We will discuss whether it is still the best current way, or if not, choose another default chunking way. In any case, I invite you to give your feedback :)

do you have any plan to implement better splitters?

By default, I would say almost no. It is very difficult to find a general solution to solve all the problems. Anyway, we have on the roadmap to move to the semantic splitter, but we don't do a lot of experiments to find out how good it is


Introduce a shell script with disabled telemetry for everything

List on https://toptout.me/ describes steps how to disable telemetry for different applications.
However, there is no single env variable that will be supported by all or most part of applications (like NO_COLOR ^1) and telemetry should be disabled separately for every application.
I suggest introducing a shell script that could be sourced to .profile and thus will disable telemetry for everything described in toptout.me. Like this:
...
HOMEBREW_NO_ANALYTICS_THIS_RUN=1
HOMEBREW_NO_ANALYTICS=1
LYNX_ANALYTICS=1
...

Have a look at ./examples, should be exactly what you're looking for 🤗.  Do you have ideas on how this can be made more discoverable?

@beatcracker thanks for the project!


[ADAM-729] Stuff TLEN into attributes.

Resolves #729. Ensures that TLEN is pushed back into the correct place in SAM on conversion back.

Looks good to me. Once Jenkins gives this a green light, I'll merge it.

Thanks, Frank!