need help on how to troubleshoot pool

this is how i used node-mysql
var mysql = require('mysql');
var connectionpool = mysql.createPool({
        host     : '<IP_ADDRESS>',
        user     : 'dbuser',
        password : 'dbuser123',
        database : 'db',
        connectionLimit : 30
});

db.getConnection(function(err, connection) { if ( !err ) {
                var q = "SELECT 1";
                connection.query(q, function(err, rows) { if ( !err ) {
                        for ( var i=0; i<rows.length; i++ ) {
                                ... process ...
                        }
                }
                connection.release();
                });
        }});

everything works ok, except that after a few hours or days, my express-based app returns no data
and i have to restart the whole app, then it will work perfect again for hours, sometimes days.
i suspect it has something to do with connections. any ideas on how to isolate this?

It sounds like you may be leaking connections somewhere. This would happen if somewhere in your code base you are .getConnection and then there is no matching .release down every code path after that--I assume that's not the sole mysql call in your whole project, correct?

yes, i got tons of calls and that's the first thing i checked when i got this behavior.
all <.getConnection> are matched with <a .release>
other than this, what other things should i look out for or how do i find the root cause?

That's the only cause that I am aware of, unfortunately. If you're not already, you can double-check that you're also using the most recent version of this library in case it's a bug as well.
You can also add this to your code: connectionpool.on('queue', function(){console.log('conn queued')}) and see if when your code is "stuck" that those messages are still being printed, which would at least isolated that it is an issue where not all connections are getting released back to the pool.

good idea, thanks doug.
will try this

i tried connectionpool.on('enqueue', function(){console.log('conn queued')})
whenever i get the lockup, i would see logs bombing out this message.
is it because connections were queued but never goes through?
so this really means i got some kind of connection leak?

is it because connections were queued but never goes through?

it means that all your calls to getConnection are queued, waiting, because there are currently connectionLimit connections (probably 30, looking at your code above).

so this really means i got some kind of connection leak?

Most likely. If you are comfortable modifying the installed source in your node_modules temporarily for debugging, the do this:

Open your local to https://github.com/felixge/node-mysql/blob/v2.5.4/lib/Pool.js#L23 and add this code: cb = function(){cb.apply(this, arguments)};cb.trace=new Error('getConnection');
Open your local to https://github.com/felixge/node-mysql/blob/v2.5.4/lib/Pool.js#L93 and add this code: cb.trace&&console.error(cb.trace.stack)

Now run your app again. Once it starts hanging, you know that at the minimum, the last trace you got leaked, as in it'll point to a call to getConnection that ran through a code path without a corresponding release call. Once you fix that, repeat again until you cannot get your app to hang any longer.

thanks doug. this will be very helpful

Hi Doug,
been trying to chase after this issue for quite some time now.
I can now verify that during the times that it is failing, i can get past .getConnection but i cannot do a .query
it just stays there doing nothing, no error as well. the client will just say socket error connection timeout
does this mean i am not having a connection leak but another issue altogether? because i can get a  connection from the pool, the query just won't go through. Is there any other reason for such behavior?
and another curious thing is, i can now predict when it will fail, around 1 hour from start time. I have monit installed monitoring http on port 80, after 1 hour of running, monit will email me saying node failed http test. and it will restart. after 1 hour, monit will email me again, and so on and so forth.

Can you post the exact version of this module you are using?

nodejs-mysql 2.5.2

crap. sorry the .getConnection also failed

now trying 2.5.4
let's wait for the next hour :)

still failed.
by the way, if i do the logging suggestion you mentioned, i get this
RangeError: Maximum call stack size exceeded
and everything stop working

Oops. Here are the corrected instructions:

Open your local to https://github.com/felixge/node-mysql/blob/v2.5.4/lib/Pool.js#L23 and add this code: var cb1=cb;cb = function(){cb1.apply(this, arguments)};cb.trace=new Error('getConnection');
Open your local to https://github.com/felixge/node-mysql/blob/v2.5.4/lib/Pool.js#L93 and add this code: cb.trace&&console.error(cb.trace.stack)

ok done that. right now i don't see anything on the logs.
time to wait for an hour

happened again after 1 hour. but all is silent.
no entries on logs

This is what i saw after the hour. connection was opened and released.
but the next .getConnection just got stuck.
Error: getConnection
at Pool.getConnection (/home/nodeuser/node_modules/mysql/lib/Pool.js:23:65)
at Pool.releaseConnection (/home/nodeuser/node_modules/mysql/lib/Pool.js:135:10)
at PoolConnection.release (/home/nodeuser/node_modules/mysql/lib/PoolConnection.js:33:15)
at Query._callback (/home/nodeuser/contacts.js:80:17)
at Query.Sequence.end (/home/nodeuser/node_modules/mysql/lib/protocol/sequences/Sequence.js:96:24)
at Query._handleFinalResultPacket (/home/nodeuser/node_modules/mysql/lib/protocol/sequences/Query.js:143:8)
at Query.EofPacket (/home/nodeuser/node_modules/mysql/lib/protocol/sequences/Query.js:127:8)
at Protocol._parsePacket (/home/nodeuser/node_modules/mysql/lib/protocol/Protocol.js:271:23)
at Parser.write (/home/nodeuser/node_modules/mysql/lib/protocol/Parser.js:77:12)
at Protocol.write (/home/nodeuser/node_modules/mysql/lib/protocol/Protocol.js:39:16)
at Socket. (/home/nodeuser/node_modules/mysql/lib/Connection.js:82:28)
I also noticed that i get a lot of this below but seldom or none of the one above except this instance before it got stuck.
Error: getConnection
at Pool.getConnection (/home/nodeuser/node_modules/mysql/lib/Pool.js:23:65)
at Object.exports.getConnection (/home/nodeuser/db_handler.js:12:17)
at /home/nodeuser/contacts.js:76:7
at IncomingForm. (/home/nodeuser/node_modules/formidable/lib/incoming_form.js:104:9)
at IncomingForm.EventEmitter.emit (events.js:92:17)
at IncomingForm._maybeEnd (/home/nodeuser/node_modules/formidable/lib/incoming_form.js:551:8)
at JSONParser.parser.onEnd (/home/nodeuser/node_modules/formidable/lib/incoming_form.js:524:10)
at JSONParser.end (/home/nodeuser/node_modules/formidable/lib/json_parser.js:34:8)
at IncomingMessage. (/home/nodeuser/node_modules/formidable/lib/incoming_form.js:129:30)
at IncomingMessage.EventEmitter.emit (events.js:92:17)
what i don't understand is, it seems like the 2nd snippet above is the release part.

From what you posted, it sounds like what I suspected: for all the times you call getConnection, your code then has some code path that doesn't call .release() in every single case. Perhaps you can post your contacts.js file?

i checked my error logs this morning, i was looking for this entry at PoolConnection.release (/home/nodeuser/node_modules/mysql/lib/PoolConnection.js:33:15)
there is no indication of release, but my out logs show that all connections are released (c.log() = console.log )
this is a snippet from contacts.js
db.getConnection is on line 76
                    db.getConnection(function(err, connection) { if ( !err ) {
                            c.log(req.ip,'api/2/contacts/status/get/ Connection Opened');
                            var q = "SELECT number num, app_status, longtitude, latitude, accuracy, UNIX_TIMESTAMP(app_lastseen) lastseen, IF(app_lastseen < DATE_SUB(NOW(),INTERVAL 6 minute),'false','true') online FROM customers WHERE CONCAT('+',number) IN (" + str + ")";
                            connection.query(q, function(err, rows) {
                                    connection.release();
                                    c.log(req.ip,'api/2/contacts/status/get/ Connection Released');
                                    if ( !err ) {
                                            str = '{"circle_users_stats":[';
                                            for ( var i=0; i<rows.length; i++ ) {
                                                    str += '{"ph":"+' + rows[i].num + '","st":' + rows[i].app_status + ',"lng":' + rows[i].longtitude + ',"lat":' + rows[i].latitude + ',"acc":' + rows[i].accuracy + ',"ls":' + rows[i].lastseen + ',"on":' + rows[i].online + '},';
                                            }
                                            str = str.slice(0,-1) + ']}';
                                            res.send(str);
                                    } else res.status(500).json({"code":c.SERVER_DATABASE_ERROR,"details":"db error","description":"DB Error"});
                            });
                    } else res.status(500).json({"code":c.SERVER_DATABASE_ERROR,"details":"db error","description":"DB Error"}) });

it's also noteworthy that i have 2 nodejs apps using this mysql module, the other one is already running for days with no problems.
memory usage of app during startup uses 40mb, before locking up, 90mb. possible memory leak?

FYI, fixed this issue.
did a complete rewrite.

Gotcha. So it sounds like there most likely was a connection leak, where .release() was not called in all possible code paths from the .getConnection().


fix: fix observable mapping logic

Resolves #51.

Codecov Report

Merging #56 (97de4a1) into main (a567a5c) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##              main       #56   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           11        12    +1
  Lines          980       903   -77
  Branches       178       148   -30
=========================================
- Hits           980       903   -77

Impacted Files
Coverage Δ

sdf_wot_converter/converters/common/jsonschema.py
100.00% <100.00%> (ø)

sdf_wot_converter/converters/sdf_to_wot.py
100.00% <100.00%> (ø)

sdf_wot_converter/converters/utility.py
100.00% <100.00%> (ø)

sdf_wot_converter/converters/wot_common.py
100.00% <100.00%> (ø)

sdf_wot_converter/converters/wot_to_sdf.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a567a5c...97de4a1. Read the comment docs.


Add Light2D blend modes for add and subtract with a maximum (clamped)

Add Light2D blend modes for add and substract with a maximum (clamped)
Fixes godotengine/godot-proposals#7026
The proposal mentions the word 'minimum' for sake of symmetry with the word 'maximum', yet I opted to only do a maximum, but for both the additive and subtractive lights. ('add_max' and 'sub_max'). So using maximum for both, as this applies to the maximal amount of adding or subtracting the light will do. A minimum would be something different, resulting in only the intersection of multiple lights being lit, which is something I have not seen anyone ask for explicitly. A minimum would be difficult to implement with regards to combinations with regular and maximum lights.
Notes on implementation:

the old add/sub modes are independent of the ordering of the lights. I made the new clamped add/sub also order independent. Mix mode was never order independent, and I did not touch it.
the clamped add/sub lights are applied separately from the regular add/sub/mix lights. This allows combining both stacking and non-stacking lights in a deterministic way. The new type of non-stacking lights are applied last. This is only relevant with regards to the mix mode, but I would not advise combining mix mode with any other mode.
you can combine clamped additive and subtractive lights. They cancel each other out. But the maxima are respected.
the performance cost for games that use 2D lights but not these new modes is non-zero but very small: for every pixel three vec3 variables are initialized to zeros and a single add with clamp is done with these zero'd variables. I did not consider it necessary to provide a shader variant. (Also, implementing that is currently outside my comfort zone =).

Tested this on my debian linux laptop with a NVIDIA Quadro T500 Mobile using the 3 render modes.

You also need to fix the typo in your commit at some point, but no rush with that as long as it's done before merge 🙂

Tested with 2 simple circle lights and there is a weird dark line in between when they are getting merged..
https://github.com/godotengine/godot/assets/2223172/9f352691-1698-4dd4-b51a-73341a5738bf
Notice how the line moves towards the light, making the sprite darker.

@Calinou I believe in your printscreens we can actually see it does not work, looking at the Add_Maximum we can clearly see that the rightmost, topmost light does not correctly blend with the one directly to the left of it:

Also same border artifacts described by @KoBeWi can be seen:

Furthermore I may be wrong but the case statements implemented in the blend compute should be inside a "switch" statement (canvas.glsl line 505):
} else if (blend_mode == LIGHT_FLAGS_BLEND_MODE_MIX) {
		color.rgb = mix(color.rgb, light_color.rgb, light_color.a);
		case LIGHT_FLAGS_BLEND_MODE_ADD_MAX: {
			vec3 delta_light = light_color.rgb * light_color.a;
			clamp_max = max(clamp_max, delta_light);
			accumulated_light_to_clamp += delta_light;
		} break;
		case LIGHT_FLAGS_BLEND_MODE_SUB_MAX: {
			vec3 delta_light = light_color.rgb * -light_color.a;
			clamp_min = min(clamp_min, delta_light);
			accumulated_light_to_clamp += delta_light;
		} break;

With regards to the screenshot about the max function supposedly not working, I quickly drew a diagram to explain what one can expect from this feature.

TL;DR: for max light blending: do not use soft lights with a gradient but lights with hard edges.
Graphs show a 1-dimensional slice of a two 2d lights, with the vertical axis being light intensity.
The top-right graph explains why there is a noticeable line in the screenshots posted by other users: the gradient makes an abrupt change. In the additive case there is a plateau and 2 softer transitions (less steep angle).
For lights with a hard edge we get the opposite: max blending gives us a nice merged plateau but additive blending gives us over-exposure (peak of 2x intensity).
Max blending, imho, is not physically realistic. It does not play well with more realistic smooth lights. It is more suited for a fog-of-war effect though.

Eh, I looked closely at my video and it seems to be optical illusion. I guess I just had wrong expectations and it doesn't solve my old proposal (I've been using a workaround since forever though, so I don't care).
It's fine if the new option requires hard edges, but this has to be better documented then. Other users might also have wrong expectations.

i dont know. I would expect this feature to be more like either we have a maximum light intensity or a minimum but not both at same time.
I think most users are looking for a combination of soft-falloff sum for the minimum and hard-falloff max for the maximum.
I need this exactly for fog-of-war. I have a custom shader atm but this means all lights need to be in the same object and are not parallel objects i.e. i would not be able to dynamically spawn lights:

I am eagerly awaiting this to be added :) It would be really useful for the fog of war I am working on. I do not know enough to be able to help contribute, sorry.


Exit gracefully when output pipe is broken

Consider an Attic command with long output, e.g. attic list <repo>::<archive> for an archive with many files.
When the output of such a command is piped to something like less, then you exit less without reaching the end of the output, you get the following backtrace:
$ attic list <repo>::<archive> | less  # and quit less immediately
Traceback (most recent call last):
  File ".../attic", line 3, in <module>
    main()
  File ".../attic/lib/python3.4/site-packages/attic/archiver.py", line 730, in main
    exit_code = archiver.run(sys.argv[1:])
  File ".../attic/lib/python3.4/site-packages/attic/archiver.py", line 720, in run
    return args.func(args)
  File ".../attic/lib/python3.4/site-packages/attic/archiver.py", line 298, in do_list
    remove_surrogates(item[b'path']), extra))
BrokenPipeError: [Errno 32] Broken pipe
Exception ignored in: <_io.TextIOWrapper name='<stdout>' encoding='UTF-8'>
BrokenPipeError: [Errno 32] Broken pipe

From the user's point-of-view, there should not be an error here. It'd be nice if this scenario is handled more gracefully.

$ python3 -c 'for i in range(1000000): print(i)' | less
Traceback (most recent call last):
  File "<string>", line 1, in <module>
BrokenPipeError: [Errno 32] Broken pipe

I pressed "q" in less after paging down once, but likely before the loop ended.

# pipe.py
import errno

try:
    for i in range(1000000):
        print(i)
except IOError as e:
    if e.errno == errno.EPIPE:
        pass

Then, I can press q in less with no errors:
$ python3 pipe.py | less


Inconsistent information as to whether guide covers "KOR / CHN / TWN New 3DS" or not

Hi. On the home page (https://github.com/Plailect/Guide/wiki) there is the paragraph:

This guide will work on New 3DS, Old 3DS, and 2DS in ALL regions on firmware 11.0.0 or below (except KOR / CHN / TWN New 3DS).

But lower down it says:

ALL regions are now compatible thanks to the new method of installation!

This seems to be inconsistent to me. Which statement is true?

Well, all regions are supported, as you can do the guide with a KOR/CHN/TWN old 3DS but since the New 3DS shipped at 9.6 in those regions the guide won't work because there is no 9.0-9.2 firmware for those regions for the New 3DS.

Thanks. Maybe it would be best to just remove the line "ALL regions are now compatible thanks to the new method of installation!" and leave the first line.

Fixed


MSI BearerAuthorizer doesn't work for the client in acr-cli/acr?

Describe the bug
I am programing to call ACR with MSI auth. But I hit 401 unauthorized error when calling GetAcrRepositories:
Error: acr.BaseClient#GetAcrRepositories: Failure responding to request: StatusCode=401 -- Original Error: autorest/azure: Service returned an error. Status=401 Code="Unknown" Message="Unknown service error" Details=[{"errors":[{"code":"UNAUTHORIZED","detail":[{"Action":"*","Name":"catalog","Type":"registry"}],"message":"authentication required, visit https://aka.ms/acr/authorization for more information."}]}]

To Reproduce
Steps to reproduce the behavior:

My code is like this

...
       msiConfig := auth.MSIConfig{
		Resource: azure.PublicCloud.ServiceManagementEndpoint,
		ClientID: "<my-msi-client-id>",
	}

	authorizer, err := msiConfig.Authorizer()
	if err != nil {
		return res, err
	}

	acrClient := acr.New(acrLoginURL)
	acrClient.Authorizer = authorizer

	lastRepository := ""
	for {
		repositories, err := acrClient.GetAcrRepositories(ctx, lastRepository, nil)
		if err != nil {
			logger.WithError(err).Error("list next batch of repositories failed")
			return res, err
		}
...

The auth package is https://github.com/Azure/go-autorest/blob/c7f947c0610de1bc279f76e6d453353f95cd1bfa/autorest/azure/auth/auth.go#L750

I can see the token getting correctly with tcpdump. And the token is valid that I can curl with it to do token exchange with my ACR.

BTW, I can succeed calling GetAcrRepositories with SP BasicAuthorizer.

       acrClient.Authorizer = autorest.NewBasicAuthorizer(
		"<my-sp-id>",
		"<my-sp-pwd>",
	)

Plus, azure cli with the msi works for me:

az login --identity -u <my-msi-resource-id>
az acr repository list -g <my-rg> -n <acr-name>

Expected behavior
Expect the MSI BearerAuthorizer working as BasicAuthorizer.
Screenshots
If applicable, add screenshots to help explain your problem.
Any relevant environment information

OS: Ubuntu 18.04.4 LTS
go version: go1.16.6 linux/amd64

 github.com/Azure/go-autorest/autorest v0.11.18
 github.com/Azure/go-autorest/autorest/adal v0.9.13
 github.com/Azure/go-autorest/autorest/azure/auth v0.4.2
 github.com/Azure/go-autorest/autorest/azure/cli v0.3.1
 github.com/Azure/go-autorest/autorest/date v0.3.0
 github.com/Azure/go-autorest/autorest/to v0.4.0

Additional context
Add any other context about the problem here.

Close this issue as it's about authentication to ACR. Please reopen the issue in https://github.com/Azure/acr/issues if you still have questions.


Wrong Address and branch name

IFSC - VIJB008007
wrong entry
VIJAYA BANK | VIJB0008007 | JAMMU AIRPORT ,JAMMU & KASHMIR | JAMMU | JAMMU | JAMMU AND KASHMIR | JAMMU AIRPORT EXTN COUNTER,JAMMU & KASHMIR |   | true | true | JAMMU | true |   | true

No longer a valid IFSC. (Missing in both the NEFT and RTGS datasets).
Closing.


HTTP/2 transaction thought it was being aborted by the client

Fixed an issue where io events were based on the state of the stream.
This showed up after a bug in is_client_state_writeable() was fixed in:
b2bddd32e555326fd4231881bde6c8d4da380965
(cherry picked from commit<PHONE_NUMBER>2f8947eb0d5843823ae37374a5fcf3)

AU check successful! https://ci.trafficserver.apache.org/job/autest-github/320/

clang format successful! https://ci.trafficserver.apache.org/job/clang-format-github/323/

RAT check successful! https://ci.trafficserver.apache.org/job/RAT-github/336/

Intel CC build successful! https://ci.trafficserver.apache.org/job/icc-github/449/

Linux build successful! https://ci.trafficserver.apache.org/job/linux-github/1911/

FreeBSD11 build successful! https://ci.trafficserver.apache.org/job/freebsd-github/2018/

clang-analyzer build failed! https://ci.trafficserver.apache.org/job/clang-analyzer-github/582/


cli and cli_test.py

final draft

@tmsincomb Random note. Looks like github thinks that you are making these commits as your old account tsincomb, not entirely sure how to fix it, but maybe worth looking into?


Some problems about the experiments.

Currently, I want to run grid-search and cross-validation on your code, but I did not know how to adjust those parameters. Would you please provide some suggestions here about parameter selection? Thank you!

The regularization hyperparameter space can be set in the model_driver.m script [through the variable lambda_range]. The rest of the hyperparameters (related to the various optimization methods) can be set using the opt struct -- you can find examples in model_driver.m and optimization_driver.m.


Documentation on how to configure the receiver's REST Transport

This PR writes documentation on how to configure the receiver's REST Transport.
Test Steps:

Include steps to test these changes

Changes

Include a comprehensive list of changes in this PR
(For web UI changes) Include screenshots/video of changes

Checklist
Testing

[ ] Tested locally?
[ ] Ran ./prime test or ./gradlew testSmoke against local Docker ReportStream container?
[ ] (For Changes to /frontend-react/...) Ran npm run lint:write?
[ ] Added tests?

Process

[ ] Are there licensing issues with any new dependencies introduced?
[ ] Includes a summary of what a code reviewer should test/verify?
[ ] Updated the release notes?
[ ] Database changes are submitted as a separate PR?
[ ] DevOps team has been notified if PR requires ops support?

Linked Issues

Fixes #issue

To Be Done
Create GitHub issues to track the work remaining, if any

#issue

Specific Security-related subjects a reviewer should pay specific attention to

Does this PR introduce new endpoints?

new endpoint A
new endpoint B

Does this PR include changes in authentication and/or authorization of existing endpoints?
Does this change introduce new dependencies that need vetting?
Does this change require changes to our infrastructure?
Does logging contain sensitive data?
Does this PR include or remove any sensitive information itself?

If you answered 'yes' to any of the questions above, conduct a detailed Review that addresses at least:

What are the potential security threats and mitigations? Please list the STRIDE threats and how they are mitigated

Spoofing (faking authenticity)

Threat T, which could be achieved by A, is mitigated by M

Tampering (influence or sabotage the integrity of information, data, or system)
Repudiation (the ability to dispute the origin or originator of an action)
Information disclosure (data made available to entities who should not have it)
Denial of service (make a resource unavailable)
Elevation of Privilege (reduce restrictions that apply or gain privileges one should not have)

Have you ensured logging does not contain sensitive data?
Have you received any additional approvals needed for this change?

I put minor grammatical/typo type corrections as suggestions to make them easy to update from the github interface if you agree. I did have a couple questions that I commented, otherwise looks good!


pipe-to command should not word wrap

Newsboat version (copy the output of newsboat -v or the first line of git show):
newsboat 2.13.0 - https://newsboat.org/
Config file (copy from ~/.newsboat/config or ~/.config/newsboat/config):
macro x pipe-to "$HOME/bin/newsboat-url-to-xclip-in"

Steps to reproduce the issue:

Open an article with

Type the keystroke to filter the article through the pipe-to command

newsboat will word wrap metadata like the "Title:" line (for example, long titles will be wrapped on to the following line)

Other info you think is relevant:
None of pipe-to's output should be word wrapped, as this makes pipe-to's output unnecessarily difficult to process with an external script.
pipe-to should just leave everything in the original article as is.

Yep, makes sense, but I have to wonder why this gets asked 11 years after pipe-to was introduced. Either there is a use-case that I fail to see, or no-one is actually using this.
If latter is true, I'm fine with breaking compatibility, but if former, I'd rather introduce a new operation or an option for the existing one. It's not clear to me how to proceed here.

As a new user of newsboat I perceived the wrapping as problematic rather quickly: I'm monitoring new research articles via feeds and would like to use pipe-to in order to parse a relevant feed item (i.e. article) and then add it to a read later list. Therefore, like diamond-lizard writes, parsing would be much easier if no wrapping would occur.
Header lines (Title, Author etc.) can of course be unwrapped again straightforwardly inside the script, but what's really annoying is that there is no way to properly unwrap the summary or content field again as one can not differentiate between a linebreak introduced by wrapping and one which was there before.
I think it would be very useful to introduce a configuration option to set the maximum textwidth and also disable wrapping entirely.

I'd like to weight in saying that I'm also a new user with the same use case, and this has equally been an issue for me.

So I looked at the code, and it turns out pipe-to wraps text at 80 characters unless text-width is set to a non-zero value, in which case that value dictates the wrap limit. So you can effectively disable the wrapping by setting a very high limit:
macro x set text-width 100500 ; pipe-to less ; set text-width 0

Instead of introducing a new setting, I'd recommend using this pattern. Also, in the next major version I'll make it so text-width applies to everything consistently.


created_at/updated_at not being set

I am maintaining some code that I inherited. Bookshelf.Model is being extended to create a base class, which all our models extend:
bookshelf.Model = bookshelf.Model.extend({

	hasTimestamps: true,

	// override so that timestamps are moments
	timestamp: function(options) {

		var now = moment();
		var keys = (_.isArray(this.hasTimestamps) ? this.hasTimestamps : ['createdAt', 'updatedAt']);
		var vals = {};

		if(keys[0]) {
			if(this.isNew(options)) {
				if(!options || options.method !== 'update') {
					vals[keys[0]] = now;
				}
			}
			else if(options && options.method === 'insert') {
				vals[keys[0]] = now;
			}
		}

		if(keys[1]) {
			vals[keys[1]] = now;
		}

		if(!vals.updatedAt) {
			console.log('no updated timestamp for ' + this.tableName);
		} else {
			console.log('updated ' + this.tableName + ' at ' + vals.updatedAt.toString());
		}

		return vals;
	}
});

var Purchase = Base.Model.extend({
	tableName: 'purchases',
},
{
	fields: {
		id: Base.Model.types.integer,
		giftMessage: Base.Model.types.string,
		createdAt: Base.Model.types.timestamp,
		updatedAt: Base.Model.types.timestamp
	}
});

But the Purchase models are being saved with null values for created_at, and updated_at, despite the fact that my warning about 'no updated timestamp for ....' is never issued, and it appears that timestamps are always set. I have also tried simply removing the 'timestamps' function override, but it makes no difference.
I can see with knex logging, that the insert statements are generated with no mention of created_at or updated_at.
What is the best way to debug this?

@sheam
You missed this.set(vals, options) see

Perfect. Thanks.
The code I inherited was overriding the timestamp function. When I upgrade Bookshelf from 8 to latest, this behaviour became broken. Add the line you suggested fixed the issue.


Invalid name on package.json (does not pass Regex)

I created a new project (File > New Project > ...) and by default the package.json came with "ASP.NET" uppercase in the name, but that is invalid, the name should be lowercase

From npm:

Some rules:

The name must be less than or equal to 214 characters. This includes the scope for scoped packages.
The name can't start with a dot or an underscore.
New packages must not have uppercase letters in the name.
The name ends up being part of a URL, an argument on the command line, and a folder name. Therefore, the name can't contain any non-URL-safe characters.

I think this is a dup of https://github.com/aspnet/Templates/issues/369
Thanks for the report!


Adding ChatBot

hi @Mayuresh-22 i want to add chatbot here can you please assign this issue to me?

Hey @divyansh2375, nice idea. Can you expose more details?

Actually i havent built chat bot before so its my first time to make a chatbot

OKay @divyansh2375, go ahead with this issue. I will assign this issue to you.

ok mayuresh

@divyansh2375 any updates? are you still working on it?

@Mayuresh-22 sir i will complete it within next 3 days actually i was busy in hackathons sorry to delay but i really want to work on this feature

No issues @divyansh2375 you can make a PR in the next 3 days. All the very best 👍


add style guide blockquote in v-if with v-for

in the vue style guide, it's an essential rule to avoid use v-if and v-for together, i thought beginners should know this.

sorry for didn't get your point, and I push a new commit use p.tip instead.

does anybody here?

@CeceWall Thanks for the ping. One extra little change and I think we're good to go :)

thanks, and i have finished that

Thanks!


Checkpoint conversion

Hey,
I am trying to evaluate a model trained with torchtitan using the lm eval harness. I am using the VLLM backend. Is there any straightforward way to convert a torchtitan model in the pytorch .pt format to, e.g., a huggingface model to be used in VLLM/lm eval harness? Within the torchtune repo, I was able to find some code for VLMs, but (a) that seems to be hardcoded for LLMs, (b) uses a new inference backend instead of e.g. relying on VLLM, and (c) I feel like there might be an easy way to convert torchtitan checkpoints rather than coming up with such an involved solution.
How did you evaluate downstream task accuracy with torchtitan models?
Thank you very much for your help.

When you save PyTorch format, is that full tensor (non-DTensor) and is saved with torch.save()?

I mean what the checkpoint.md docs of torchtitan call dcp_to_torch. So the .pile generated from that, which is not - at least natively - supported by hf.

You can take take a look at convert_llama_weights_to_hf.py


portalocker error: FileNotFoundError: [Errno 2] No such file or directory

This is autogenerated. Please review and update as needed.
Describe the bug
Command Name
az account get-access-token
Errors:
The command failed with an unexpected error. Here is the traceback:
[Errno 2] No such file or directory
Traceback (most recent call last):
  File "/opt/az/lib/python3.6/site-packages/knack/cli.py", line 231, in invoke
    cmd_result = self.invocation.execute(args)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 657, in execute
    raise ex
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 720, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 691, in _run_job
    result = cmd_copy(params)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 328, in __call__
    return self.handler(*args, **kwargs)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
    return op(**command_args)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/profile/custom.py", line 75, in get_access_token
    tenant=tenant)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/_profile.py", line 381, in get_raw_token
    credential = self._create_credential(account, tenant)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/_profile.py", line 588, in _create_credential
    return identity.get_user_credential(username_or_sp_id)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/auth/identity.py", line 182, in get_user_credential
    return UserCredential(self.client_id, username, **self._msal_app_kwargs)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/auth/msal_authentication.py", line 41, in __init__
    accounts = self.get_accounts(username)
  File "/opt/az/lib/python3.6/site-packages/msal/application.py", line 872, in get_accounts
    accounts = self._find_msal_accounts(environment=self.authority.instance)
  File "/opt/az/lib/python3.6/site-packages/msal/application.py", line 912, in _find_msal_accounts
    query={"environment": environment})
  File "/opt/az/lib/python3.6/site-packages/msal_extensions/token_cache.py", line 53, in find
    with CrossPlatLock(self._lock_location):
  File "/opt/az/lib/python3.6/site-packages/msal_extensions/cache_lock.py", line 29, in __enter__
    file_handle = self._lock.__enter__()
  File "/opt/az/lib/python3.6/site-packages/portalocker/utils.py", line 199, in __enter__
    return self.acquire()
  File "/opt/az/lib/python3.6/site-packages/portalocker/utils.py", line 161, in acquire
    fh = self._prepare_fh(fh)
  File "/opt/az/lib/python3.6/site-packages/portalocker/utils.py", line 194, in _prepare_fh
    fh.truncate(0)
FileNotFoundError: [Errno 2] No such file or directory

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Put any pre-requisite steps here...
az account get-access-token --resource {} --subscription {} -o={}

Expected Behavior
Environment Summary
Linux-<IP_ADDRESS>-microsoft-standard-WSL2-x86_64-with-debian-bullseye-sid, Ubuntu 20.04.3 LTS
Python 3.6.10
Installer: DEB

azure-cli 2.30.0

Extensions:
azure-devops 0.20.0

Also, wanted to notify that I am using terragrunt for deploying azure resources and after I switched from 2.29.2 to 2.30.0 using this manual https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-linux?pivots=apt#install-specific-version it is showing similar error like above. Seems to me that the new az cli version is broken somehow

@jiasli for awareness

The error is from portalocker.
https://github.com/WoLpH/portalocker/blob/3025e66895ce819d806ac7201041af9d0b4f67bb/portalocker/utils.py#L298
            fh.seek(0)
            fh.truncate(0)

It is pretty weird that seek succeeds, but truncate failed.
Could you check if msal_token_cache.json.lockfile was deleted during execution?
+@rayluo

Is this a steady repro? Or are you running multiple az instances concurrently?

The error is from portalocker.
https://github.com/WoLpH/portalocker/blob/3025e66895ce819d806ac7201041af9d0b4f67bb/portalocker/utils.py#L298
            fh.seek(0)
            fh.truncate(0)

It is pretty weird that seek succeeds, but truncate failed.
Could you check if ~/.azure/msal_token_cache.json.lockfile was deleted during execution?
+@rayluo

portalocker author @wolph for awareness

Could it somehow be a race condition where something else is either deleting the file? It looks like the cache_lock.py code can remove the lock file on exit which might be related: https://github.com/AzureAD/microsoft-authentication-extensions-for-python/blob/26240cbc092e0fd28badf52753888051db3a4cec/msal_extensions/cache_lock.py#L65-L74

Possibly. Currently MSAL Extensions indeed creates and deletes that lock file frequently.
Does portalocker expect the lock file to remain available most of the time?
Could portalocker be refactored so that any failure during this preparation can be put inside the retry logic?

Does portalocker expect the lock file to remain available most of the time?

As soon as the lock is acquired, portalocker does not expect the lock to disappear anymore. That seems a reasonable assumption to me since nothing else should be able to acquire the lock at that point.

Could portalocker be refactored so that any failure during this preparation can be put inside the retry logic?

Yes, that's certainly an option. But I think there might be an underlying bug somewhere that is causing this. This scenario should never occur if locking works properly.

Is this a steady repro? Or are you running multiple az instances concurrently?

I am not 100 % sure, but I would say it is running multiple 'az' instances concurrently, because when running "terragrunt run-all ..." it goes to multiple directories and as far I suspect executes az token command at the same time

Does portalocker expect the lock file to remain available most of the time?

As soon as the lock is acquired, portalocker does not expect the lock to disappear anymore. That seems a reasonable assumption to me since nothing else should be able to acquire the lock at that point.

Could portalocker be refactored so that any failure during this preparation can be put inside the retry logic?

Yes, that's certainly an option. But I think there might be an underlying bug somewhere that is causing this. This scenario should never occur if locking works properly.

@wolph, yes, I understand that the MSAL EXtension's usage pattern of frequently creating and deleting the locker file sounds somewhat unusual. MSAL EX chose to do so, because it wants to inter-operate with its counterparts written in other languages, which collectively chose to use the presence or absence of the lock file as part of the locking mechanism.
In terms of portalocker, it seems there would be no downside if we put that truncate(0) into the retry logic?

In terms of portalocker, it seems there would be no downside if we put that truncate(0) into the retry logic?

There's no real downside, the only "downside" I see is that it could hide these errors. This scenario should not occur so I do believe it should warrant a warning that your lock might not be valid anymore.
Once the lockfile has disappeared, your lock has disappeared as well and anyone else could lock in the mean time.

In terms of portalocker, it seems there would be no downside if we put that truncate(0) into the retry logic?

There's no real downside, the only "downside" I see is that it could hide these errors. This scenario should not occur so I do believe it should warrant a warning that your lock might not be valid anymore.
Once the lockfile has disappeared, your lock has disappeared as well and anyone else could lock in the mean time.

Thank you, @WoLpH , I agree with your assessment. I am currently trying to make some change from our side. I'll ping you again if we need your attention.

Hello, any updates on that?

Hello, any updates on that?

We are still working on it. But perhaps you won't mind try this experimenting adjustment? https://github.com/Azure/azure-cli/issues/20273#issuecomment-971409849
If you try it, please report back here in this issue to let us know the result. Thanks!

@edvinas31 , this solution should work. Please reopen if that does not work for you.

@edvinas31 , this solution should work. Please reopen if that does not work for you.

Good morning @rayluo . Tried to use your mentioned solution and ran following line on my local computer "python3 -m pip install -U msal-extensions" (used wsl2 ubuntu) and still got the same issue:
.... │ Error: Error obtaining Authorization Token from the Azure CLI: Error parsing json result from the Azure CLI: Error waiting for the Azure CLI: exit status 1: ERROR: The command failed with an unexpected error. Here is the traceback: │ ERROR: [Errno 2] No such file or directory ...
This happens when using 2.31.0 version

On Ubuntu, the quick fix should be
sudo /opt/az/bin/python3 -m pip install --upgrade msal-extensions

As for [Errno 2] No such file or directory, could you share the full call stack?

I have ran command:
sudo /opt/az/bin/python3 -m pip install --upgrade msal-extensions
However I got:
│ Error: Error obtaining Authorization Token from the Azure CLI: Error parsing json result from the Azure CLI: Error waiting for the Azure CLI: exit status 1: ERROR: The command failed with an unexpected error. Here is the traceback:
│ ERROR: Ran out of input
│ Traceback (most recent call last):
│   File "/opt/az/lib/python3.6/site-packages/knack/cli.py", line 231, in invoke
│     cmd_result = self.invocation.execute(args)
│   File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 658, in execute
│     raise ex
│   File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 721, in _run_jobs_serially
│     results.append(self._run_job(expanded_arg, cmd_copy))
│   File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 692, in _run_job
│     result = cmd_copy(params)
│   File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 328, in __call__
│     return self.handler(*args, **kwargs)
│   File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
│     return op(**command_args)
│   File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/profile/custom.py", line 77, in get_access_token
│     tenant=tenant)
│   File "/opt/az/lib/python3.6/site-packages/azure/cli/core/_profile.py", line 383, in get_raw_token
│     credential = self._create_credential(account, tenant)
│   File "/opt/az/lib/python3.6/site-packages/azure/cli/core/_profile.py", line 593, in _create_credential
│     return identity.get_user_credential(username_or_sp_id)
│   File "/opt/az/lib/python3.6/site-packages/azure/cli/core/auth/identity.py", line 209, in get_user_credential
│     return UserCredential(self.client_id, username, **self._msal_app_kwargs)
│   File "/opt/az/lib/python3.6/site-packages/azure/cli/core/auth/identity.py", line 84, in _msal_app_kwargs
│     Identity._msal_http_cache = self._load_msal_http_cache()
│   File "/opt/az/lib/python3.6/site-packages/azure/cli/core/auth/identity.py", line 113, in _load_msal_http_cache
│     persisted_http_cache = pickle.load(f)
│ EOFError: Ran out of input
│ To open an issue, please run: 'az feedback

This is another problem caused by ~/.azure/msal_http_cache.bin. I am not very clear how it happens.
Could you remove ~/.azure and try again?

Removed ~/.azure and tried again. Same issue happens:
│ Error: unable to get authorization token for resource manager: Error obtaining Authorization Token from the Azure CLI: Error parsing json result from the Azure CLI: Error waiting for the Azure CLI: exit status 1: ERROR: The command failed with an unexpected error. Here is the traceback:
│ ERROR: Ran out of input
│ Traceback (most recent call last):
│   File "/opt/az/lib/python3.6/site-packages/knack/cli.py", line 231, in invoke
│     cmd_result = self.invocation.execute(args)
│   File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 658, in execute
│     raise ex
│   File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 721, in _run_jobs_serially
│     results.append(self._run_job(expanded_arg, cmd_copy))
│   File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 692, in _run_job
│     result = cmd_copy(params)
│   File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 328, in __call__
│     return self.handler(*args, **kwargs)
│   File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
│     return op(**command_args)
│   File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/profile/custom.py", line 77, in get_access_token
│     tenant=tenant)
│   File "/opt/az/lib/python3.6/site-packages/azure/cli/core/_profile.py", line 383, in get_raw_token
│     credential = self._create_credential(account, tenant)
│   File "/opt/az/lib/python3.6/site-packages/azure/cli/core/_profile.py", line 593, in _create_credential
│     return identity.get_user_credential(username_or_sp_id)
│   File "/opt/az/lib/python3.6/site-packages/azure/cli/core/auth/identity.py", line 209, in get_user_credential
│     return UserCredential(self.client_id, username, **self._msal_app_kwargs)
│   File "/opt/az/lib/python3.6/site-packages/azure/cli/core/auth/identity.py", line 84, in _msal_app_kwargs
│     Identity._msal_http_cache = self._load_msal_http_cache()
│   File "/opt/az/lib/python3.6/site-packages/azure/cli/core/auth/identity.py", line 113, in _load_msal_http_cache
│     persisted_http_cache = pickle.load(f)
│ EOFError: Ran out of input
│ To open an issue, please run: 'az feedback'
│
│   with provider["registry.terraform.io/hashicorp/azurerm"],
│   on terraform_config.tf line 13, in provider "azurerm":
│   13: provider "azurerm" {

Moving to a new issue #20707.


Rails/Validation cannot autocorrect validation methods with parenthesis properly

If you have a code like:
validates_presence_of(:user, :address)

The autocorrection of Rails/Validation will generate code with incorrect syntax.
Expected behavior
validates :user, :address, presence: true

Actual behavior
validates(:user, :address), presence: true

Steps to reproduce the problem

inside an ActiveRecord class, insert line validates_presence_of(:user, :address)
call rubocop -a

RuboCop version
$ [bundle exec] rubocop -V
0.65.0 (using Parser <IP_ADDRESS>, running on ruby 2.5.3 x86_64-darwin16)

Thanks for the feedback. I opened a PR #6807.


⚠ GitLab Server has degraded performance

In 6bcd9b2, GitLab Server (https://gitlab01.its-telekom.eu) experienced degraded performance:

HTTP code: 200
Response time: 11001 ms

Resolved: GitLab Server performance has improved in 170a16b.


Facing Issue to the parameter in ordering as I passed to "remote-url" & remote-url-request-formatter attribute

Hi,
I am facing issue when I passing the querystring value with "remote-url-request-formatter"..
<angucomplete-alt ng-repeat="(key, value) in vm.homelabel | objectByKeyValFilter:'LblId':'txtDestinationSearch'"
                                                     placeholder="{{value.TextMessage}}"
                                                     pause="300"
                                                     selected-object="vm.selected_destination"
                                                     remote-url="http://localhost:8096/api/Home/GetSearchResult" ,
                                                     remote-url-request-formatter="vm.remoteUrlRequestFn"
                                                     remote-url-data-field="Name"
                                                     search-fields="Name,DisplayName"
                                                     title-field="Name"
                                                     minlength="1"
                                                     input-class="form-control small-input"
                                                     match-class="highlight"
                                                     focus-in="vm.focusIn()"
                                                     focus-out="vm.focusOut()"
                                                     id="txtdestinationsearch" autocomplete="off" />

in JS File with passing dynamic query string value for search..
vm.remoteUrlRequestFn = function (str) {
           debugger;
           return { "culture": vm.lang, "keyword": str, "countryId":  "in" };
       };

I want to call API with this query string order
http://localhost:8096/api/Home/GetSearchResult?CultureCode=en&Keyword=par&CountryID=in
but in above case when it call remote-url-request-formatter at that time its order changed as below
http://localhost:8096/api/Home/GetSearchResult?countryId=in&culture=en&keyword=par
You can see both API URL with query string parameters. how can I make in ordered..
Thanks

That's not possible with angucomplete-alt.
Query params are usually treated as a hash, so they don't keep the order.
If you are implementing the server, you should change your server implementation.

ok @ghiden  thank you.
I will change order of parameter in my service parameter and methods.
Thank you.


Enable passing ssl certificates when creating AuthenticatedClient()

Extend the ClientOpts struct to support the addition a string feild, CACert. Then use the cert passed in all following openstack authentication requests.

@iamemilio Please see https://github.com/gophercloud/utils/pull/99 instead. This will allow an http client to be configured with the appropriate SSL configuration. As an example of that, see here: https://github.com/gophercloud/utils/blob/master/terraform/auth/config.go
Adding SSL data directly to the client will create a tight coupling of the http client and openstack client which we want to avoid in order to allow better flexibility across applications.

Understood, thanks for informing me before I wasted a lot of time and effort :). This would work just as well for my use case. If there is anything I can contribute to help you get #99 merged please let me know.

It looks like I need to tweak Travis and possibly drop support for some old Go versions, but other than that, I don't mind merging #99 at all. I should be able to get to that in the next day.

@iamemilio #99 has been merged.
To take advantage of it, create and configure an http.Client struct with TLS via http.Transport. If you aren't familiar with how to do this, it's all native to Golang and nothing specific to Gophercloud, so any external resource can help. We have some examples here:

https://github.com/gophercloud/utils/blob/master/acceptance/openstack/clientconfig/clientconfig_test.go#L64-L73
https://github.com/gophercloud/utils/blob/master/terraform/auth/config.go#L164-L214

But they're also doing some extra things like configuring a RoundTripper to print REST debug info.
But once you have your http.Client set up correctly, you can then pass it via the new ClientOpts field.
The next part gets a little tricky. If you're using the NewServiceClient convenience function, then everything is handled automatically. But if the application you're working on isn't able to leverage NewServiceClient (which is totally fine - NewServiceClient is only meant to handle common cases), then you'll need to do the next step yourself, but fortunately it's simple: just do this:
https://github.com/gophercloud/utils/blob/master/openstack/clientconfig/requests.go#L743-L746
Basically just pass opts.HTTPClient to the provider client's HTTPClient field.
Once that is in place, then all HTTP requests will leverage the SSL configuration.
If you need the SSL configuration for SSL/TLS client authentication, I don't think NewServiceClient will work since the provider client is created before HTTPClient is passed - I might need to revisit that. Instead, try following closely with the Terraform auth code linked above (even outside the highlighted areas). The highlighted chunk of code began specifically to support SSL/TLS client auth, so it should be done in a way where the client is set up correctly before any initial HTTP request is made, thus allowing a provider client to be authenticated properly. At least, no one has reported it doesn't work :slightly_smiling_face:
Please let me know if you have any questions or run into any problems.


Update View command functionality

This commit will fix the following functionality issues:

View Window closes when "exit" is called
View Window now updates when the address book is edited.

LGTM


Remove Understanding Power Platform Licensing Article

Article issue
@bigpix2000 @LuiseFreese
We've been asked to remove the Understanding Power Platform Licensing article. I'm going to do a PR this morning to take it out of the repo. I don't think we need to discuss this here, but I wanted to let you know.
@pamgreen-msft

@sympmarc oh dear, that sounds major.  please let me know as soon as possible at the appropriate place. Thank you


standardize_X issue in model.py

Hi,
This is a great package. However, I found it breaks on the example with the following message.
$ python examples/dbn_example.py
Using Theano backend.
Traceback (most recent call last):
  File "examples/dbn_example.py", line 14, in <module>
    from keras_extensions.dbn import DBN
  File "//anaconda/lib/python2.7/site-packages/keras_extensions-0.1-py2.7.egg/keras_extensions/dbn.py", line 13, in <module>
  File "//anaconda/lib/python2.7/site-packages/keras_extensions-0.1-py2.7.egg/keras_extensions/models.py", line 1, in <module>
ImportError: cannot import name standardize_X

I'm using this keras version:
In [4]: keras.__version__
Out[4]: '1.0.2'

Best,
E.W.

Hi,
Thanks for reporting.
Keras quite frequently breaks backwards compatibility; which is OK, but this package tends to break because it contains some classes which are modified copies of Keras classes.
I don't use Keras myself currently; so I'm not very motivated to keep this package up-to-date at all times.
I recommend you either use an older version of Keras (don't know exactly which, but pre-1.0 for sure; check date of last commit). Or try to fix the issue yourself (and submit a PR :) ). If you look at the code that generates the error there should be a comment referencing some Keras class which you can have a look at.

Hi !
After I encountered the same issue with standardize_X, I have successfully run dbn_example.py with keras 0.3.0. This may be some helpful information, even if it comes late.

Hi !
I try to run the example but it breaked on No.82 line: train_model.fit(X_train, batch_size, nb_epoch, verbose=1, shuffle=False, [callbacks=callbacks)]. And the error is as followed:
TypeError: ('Bad input argument to theano function with name "D:\PYTHON27\lib\site-packages\keras\backend\theano_backend.py:915"  at index 0(0-based)', 'Wrong number of dimensions: expected 2, got 1 with shape (100L,).')
Best,
H.M

Hi,
I try to installed keras 0.3.0, but shows in cmd for keras.version  as 0.2.0. Moreover tried with 0.3.1 version it shows following error as 'ImportError: cannot import name downsample'.
while i do this above process, unistalled all keras version packages.
Thanking you in advance...
Regards,
Koteswar

May be its too late, but it could help someone in future. This works with keras 0.20 or 0.30 with theano 0.8.0. In later versions of theano you get 'cannot import name downsample' as downsample module is moved.

Still having issues importing standardize_X after installing keras 0.2.0 and theano 0.8.0.
@miaohang123 it looks like you're using python 2.7 (your native system python) maybe try with Python 3 with a virtual envt?
Should I solve this issue, I'll report what I did shortly.


Fix: better handling of id replacement

Description
Wasn't supporting subflows and groups. This adds a more universal solution to the ID mapping we do to prevent Node-RED clashes on unchanged nodes.

where the id exists as some other value in a node or API key or something.

even if that's the case, we don't want this new flow pointing to old flow instances, they all need updating.

Will let you decided Joe (I may be being over cautious)

We proceed as is, if new users report problems, we handle them


update AI version

Codecov Report

Merging #846 into develop will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             develop     #846   +/-   ##
==========================================
  Coverage      44.55%   44.55%
  Complexity       765      765
==========================================
  Files            134      134
  Lines           5169     5169
  Branches         742      742
==========================================
  Hits            2303     2303
  Misses          2692     2692
  Partials         174      174

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update eee7df1...7fc7b0a. Read the comment docs.

Do we need to update the ai version of plugin lib/common as well?


machine url panics

Description of problem:
If no machine is active, machine url panics
Machine version:
Running latest of master of machine
Environment details:
Running machine on Mac OSX Yosemite
How reproducible:
Very
Steps to Reproduce:

Get into a state where you have no active machine. Easiest is to just not create a machine. But I think deleting the active machine should also do the trick.
Run machine url with no machine/host specified

Actual Results:
machine panics:
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0xe90c2]

goroutine 1 [running]:
github.com/docker/machine/hosts.(*Host).GetURL(0x0, 0x0, 0x0, 0x0, 0x0)
	/Users/cjellick/go/src/github.com/docker/machine/hosts/host.go:249 +0x92
main.cmdUrl(0xc20809c180)
	/Users/cjellick/go/src/github.com/docker/machine/commands.go:377 +0x41
github.com/codegangsta/cli.Command.Run(0x699510, 0x3, 0x0, 0x0, 0x7017b0, 0x18, 0x0, 0x0, 0x0, 0x0, ...)
	/Users/cjellick/go/src/github.com/docker/machine/Godeps/_workspace/src/github.com/codegangsta/cli/command.go:101 +0xe42
github.com/codegangsta/cli.(*App).Run(0xc20809c000, 0xc20800a000, 0x2, 0x2, 0x0, 0x0)
	/Users/cjellick/go/src/github.com/docker/machine/Godeps/_workspace/src/github.com/codegangsta/cli/app.go:130 +0xb81
main.main()
	/Users/cjellick/go/src/github.com/docker/machine/main.go:36 +0x3ea

goroutine 5 [syscall]:
os/signal.loop()
	/usr/local/Cellar/go/1.4/libexec/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
	/usr/local/Cellar/go/1.4/libexec/src/os/signal/signal_unix.go:27 +0x35

goroutine 6 [runnable]:
text/template/parse.lexText(0xc208068100, 0x7c2580)
	/usr/local/Cellar/go/1.4/libexec/src/text/template/parse/lex.go:228 +0x37b
text/template/parse.(*lexer).run(0xc208068100)
	/usr/local/Cellar/go/1.4/libexec/src/text/template/parse/lex.go:198 +0x5d
created by text/template/parse.lex
	/usr/local/Cellar/go/1.4/libexec/src/text/template/parse/lex.go:191 +0x1ac
~/go/src/github.com/docker/machine$ mvim main.go
~/go/src/github.com/docker/machine$ mvim /usr/local/Cellar/go/1.4/libexec/src/text/template/parse/lex.go

Expected Results:
Graceful failure and meaningful user-friendly error message.

This happens on all commands actually...

Indeed.  Dupe of #222 I think.

@cjellick Is this something you're working on?

@sthulb I started working on it on Friday, but got into a bit of yak shaving.

@cjellick OK... I think I narrowed it down to a problem with the first return in GetActive from the store. It just returns nil, nil

@sthulb your PR (#262) cover it then? I'd be duplicating effort if I pursued a fix further at this point, right?

@cjellick It should cover it.

Closing as #262 is merged.


What about adding mxnet to pip ?

What about adding mxnet to pip so that we can easily install mxnet?

Try pip install mxnet and pip install mxnet_gpu.
We have a experimental version that support linux (ubuntu and maybe some other newer distros) and mac os.

For GPU support, I've published mxnet-cu80 and mxnet-cu75 which supports CUDA8.0 and 7.5 respectively to replace mxnet-gpu. Both versions use cuDNN5.1.


[Unity] Responses are handled in main thread

Copied from external bug report:
Asynchronous API calls like FindAsync can and do block the main thread when the response is received by the client. Operations like parsing the response data are supposed to happen in background threads, as described in the Tasks API, but they seem to be happening on the main thread anyway.
To isolate this, I created a bare bones project in Unity that calls FindAsync with no continuation. If the data requested takes a long time to parse (I've seen 3.5 seconds for a large string with a lot of escape characters), this blocks the main thread, preventing rendering and user input until the parsing is completed.
In the profiler it appears that tasks are calling their continuations directly instead of spawning separate tasks. This may be an optimization to prevent excess thread creation, but there need to be safeguards in place to ensure that operations that may take a long time to complete cannot run on the main thread.

We are seeing this in our Xamarin iOS app. When the app loads a lot of data from Parse it freezes for a few seconds while it is parsing the data especially on older phones.
Would like if we can speed up parsing (maybe use Json.net) or move the parsing to a background thread.

This issue should be solved due to improved support for async-await by Unity.
If you encounter any similar problems with the latest version of the SDK feel free to create a new issue to discuss possible solutions.


feat: adding reactive html export

Big fan of the latest islands feature!
This PR extends the _server and _cli export functionality to return an interactive HTML using marimo islands.
CLI usage (note CLI args available in regular html export are currently ignored):
marimo export html notebook.py --reactive --no-include-code -o notebook-reactive.html

Low-level python usage (e.g. what I use to define ObservableHQ data-loaders):
from marimo._server.export import run_app_then_export_as_reactive_html
from marimo._utils.marimo_path import MarimoPath
from bs4 import BeautifulSoup
import asyncio

path = MarimoPath("src/data-files/notebook.py")
html, _ = asyncio.run(
    run_app_then_export_as_reactive_html(
        path, include_code=False
    )
)

soup = BeautifulSoup(html, features="html.parser")
body = soup.body.decode_contents().strip()
print(body)

Happy to include CLI tests if this looks good!

I have read the CLA Document and I hereby sign the CLA.

recheck

I like this - we should probably expose a low-level python api that isn't private (don't want to break APIs) and hopefully the smallest api (just a path?)

or maybe we don't need the low-level api? would you still use it, if this CLI addition was added?

I would probably still call this from the low-level api since I suspect most SSGs (e.g. Observable Framework) won't handle the html as is, and one might as-well manipulate it further with python (e.g. using BeautifulSoup as above).

Thanks for the prompt response!

I tried using the CLI export on the intro.py tutorial and noticed that some styles were off (notebook width, codemirror editors, and KaTeX didn't render are a few things I noticed). marimo export should look good out of the box.

This uses MarimoIslandStub.render() directly - but agreed we should perhaps look at the tweaking some inline css styles or similar. FWIW, when I use this to embed on Observable KaTeX seems to work?

If we add a CLI, we could omit the low-level API. When integrating with Observable, you could just run the marimo CLI (either in a Python script using subprocess, or in a shell script, etc), unless I am missing something.

Sure yeah - to clarify, I'm perfectly happy calling the low-level "private" API as above (which presumably the CLI will call as-well) or, like you said, running the CLI tool and then piping the output to a different process for post-processing.

It seems odd that cli-args is ignored when --reactive is passed. That sort of suggests to me that this should be its own export option, eg marimo export wasm or marimo export reactive-html.

marimo export reactive-html was my initial thought, and then I realized I was essentially copying a lot of the html code and switched to this. Happy to go back if you think it's cleaner.

Cool! So Observable data loaders use require a *.html.py* right? Or since invoking from marimo, are you using a *.html.sh?

@dmadisetti Indeed, the simplest case would be a data-loader called marimo-intro-cli.html.sh with the contents
#!/bin/bash
marimo export html src/data-files/intro-notebook.py --reactive --no-include-code

which one can call as follow in an Observable Framework md file:
```js
const marimo_html = FileAttachment("data/marimo-intro-cli.html").html();
```

 ${marimo_html.body}

Added the repo on git, if you wanted to have a look. Deployed site can be found here.

If the data loader just goes off the shebang, I think you should be able to do
#!/user/bin/env marimo  export html --reactive --no-include-code

import marimo as mo
# normal app here
# ...

To clarify, I don't necessarily need this PR for my purposes - I'm perfectly happy using the snippet I added in export.__init__.py as my Observable dataloader. This was mostly because I wanted to contribute to the codebase and figured others might also like a reactive HTML export!
Pushed the changes I was working on before your comment @mscolnick - I think it might capture atleast some of @akshayka concerns, if you want to check it out.
Regardless of the cli, a static MarimoIslandGenerator.from_file() does sound like a good idea.

I think this makes sense; islands are a little bit hidden otherwise. Regarding the limited format, I think this and normal export HTML - could be expanded to take a template, but that probably doesn't need to be added until it's needed.

@gvarnavi , thanks for clarifying, and thanks for contributing back to us! We definitely appreciate it :)
@dmadisetti, ok, good point -- wasm export is hard to find/low-level with just islands.
If I understood the above discussion, it will be sufficient to just add reactive HTML export to the CLI; that will enable both gvarnavi's use case as well as let less advanced users benefit from WASM exports. If this is correct, then I support adding reactive HTML export to the CLI.
@gvarnavi, I took a look and the styling does look better -- thank you. Three comments:

Margins. Below I've pasted a side-by-side comparison of marimo run intro.py (left) and marimo export html --reactive --no-include-code intro.py (right). Notice that the right-hand export is missing vertical margins. Is this possible to fix?

Reactivity. When testing locally, the exported HTML is not actually reactive. Is this just a me problem? Here's what I see in the console:

Not allowed to load local resource: blob:null/049a4c1b-06df-4771-a0fa-50275133aa6b

CLI Args. Why not support CLI args? If it's too hard to support, we should at least raise an Exception if cli_args are non-empty when exporting reactive HTML.

Oh yikes. # 3 might be from a work around I did to get a local cross site Web Worker. Never tested from a non-served file. This might work in islands bridge: https://gist.github.com/walfie/a80c4432bcff70fb826d5d28158e9cc4
But will probably need to tinkering. I know Myles has taken a pass at that web worker too: https://github.com/marimo-team/marimo/blob/69830587f343f839d777155a870056d010044ea6/frontend/src/core/islands/bridge.ts#L51

@dmadisetti @akshayka , im pretty sure you can't run a webworker from file:// (or at least ours) - needs to be a hosted server

(cc @gvarnavi)

im pretty sure you can't run a webworker from file:// (or at least ours) - needs to be a hosted server

Oh shoot -- I wasn't aware of that. I guess that's for security?
If that's the case we might as well not include this in the CLI, and just recommend that people use marimo.io or islands -- can already imagine all the questions we'd get from our users about why their reactive exports aren't working.
@mscolnick I'll leave the decision up to you on how to proceed.

im pretty sure you can't run a webworker from file:// (or at least ours) - needs to be a hosted server

Ah indeed, hadn't realized either since I was mostly testing directly on Observable - which is hosted.
Hmm, perhaps the easiest solution then would be a static MarimoIslandGenerator.from_file()  class method, and some additional documentation/examples pointing users to it, as @mscolnick suggests -- I can add that sometime tomorrow.

If that's the case we might as well not include this in the CLI, and just recommend that people use marimo.io or islands -- can already imagine all the questions we'd get from our users about why their reactive exports aren't working.

If the major concern is communication to users, then I think this could be mitigated with a alert dialog with something like:

[!CAUTION]
marimo requires Web Workers to run reactively, and as such marimo islands will not work with local files. Please host this file.

Because not allowing export doesn't get rid of the problem that islands won't work locally. But addressing enables this export, and documents the edgecase

Pardon the delay, just got round to this.
Added a static MarimoIslandGenerator.from_file() method per @mscolnick suggestion. This indeed simplifies the islands html generation a fair bit. e.g. I use the following snippet for Observable.
import asyncio

from marimo import MarimoIslandGenerator

generator = MarimoIslandGenerator.from_file("src/data-files/dislocation-fields.py")
app = asyncio.run(generator.build())
body = generator.render_body()
print(body)

Note: the new render_body() and render_html() functions are nice-to-have but not necessary utility functions.
Also, I've kept the cli export for now (and switched it to using the simpler static method above), until we reach a consensus on whether or not to include it. I like @dmadisetti's warning idea -- perhaps this should be implemented at the islands js file though? As in check if the file is served from a local file (perhaps using window.location.protocol or similar), and issue a warning if so?

Great, thanks for the review @mscolnick!

force merging - looks like unrelated tests are failing on main


Suggestion: Add a comparison with Marp

Suggestion: Add a comparison with Marp in the README, as Marp is a well-known similar project that many people are familiar with.

Good idea! Currently I don’t view moffee as feature-completed. Many useful features and optimizations are being implemented. I will add a comparison section after a few more updates.
2024 年 8 月 16 日星期五 8:13 上午, Huanyu @.***(mailto:2024 年 8 月 16 日星期五 8:13 上午, Huanyu < 来信：

Suggestion: Add a comparison with Marp in the README, as Marp is a well-known similar project that many people are familiar with.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.Message ID: @.***>


Docker Pull Job Type Images

Add a task to the job execution task chain that performs a Docker pull against the job type Docker image to prevent timeouts during Mesos task staging. Consider how this should be tracked in the job execution model (right now there are 3 tasks: pre, job, and post). Should a job execution also include a pull task, or should there be a TTL or some other mechanism to indicate that fresh pulls are needed for a job type/node combo.
Also update tasks to request reconciliation after a much shorter period (30 seconds?) if they have been launched, but are still not running (i.e. still in staging).
Consider removing forcePull for the image on the job task.
Also consider a health check to see if the Docker registry is reachable. If not, should jobs stop running? Should there be a master scheduler field that dictates whether the registry is required at the moment? If not required, all jobs could run without pulls (what about nodes with old/non-existent images?) and health checks for the registry be ignored.

This issue is now OBE.


Discord timestamp generator

Implemented as a basic HTML/CSS/JS web page with form input to generate Discord timestamp strings in a variety of formats. Should look like this, basically: https://alexholderdeveloper.github.io/DiscordTimestampGenerator/src/index.html

Rules:-
1)Follow @shubham7668 and add star in this repository
3)Add snapshots of working project
Follow all rules in order to get hacktoberfest-accepted tag

Followed and starred.
Snapshot was linked in the PR description, but a screenshot if you need it:


Update description and README for clarity

These details are abstract, from a time when Graphula was meant to be abstracted from any particular storage mechanism. We've since decided being tied to persistent is a valuable simplification. We should update our package description to indicate this coupling, and use more compelling, persistent-specific examples in the README.

Is a name change in order as well? We've talked about making it a persistent-.

I'm not against, but I probably wouldn't be able to do the work for while. So it'll just need a new champion.

Lots of improvements in 070645e4fe697f15bd088ee59c6e1c4704684f3d, and I will tweak the package.yml as part of next release.


XcodeProjectInterpreter violates our "no timeouts" rule

If these processes are idempotent, then I suggest that we change this to not kill the first process, but to show a message after a few seconds saying "This seems to be taking longer than usual" and then start a second process in parallel, to see if it returns faster, and if it does, kill the first one. If the second takes more than a minute, kill it then try a third one, etc, leaving still the first one running, in case it's just being super slow.

Looks like there are still a lot of timeouts in the code for XcodeProjectInterpreter:
https://github.com/flutter/flutter/blob/c23d9cd4ce85aff95360ac0b61cd93344e123f92/packages/flutter_tools/lib/src/ios/xcodeproj.dart#L234
/cc @jmagman FYI


AudioContext-only functions are still patched on BaseAudioContext

The createMediaElementSource(), createMediaStreamSource() and createMediaStreamDestination() factory functions get still patched on the BaseAudioContext prototype. But some time ago they have been moved to the AudioContext prototype.
I think this Stack Overflow question is describing the symptom of that bug.

Are you interested in submitting a PR for this? :)


python311Packages.coffea: 2024.1.0 -> 2024.2.1

Diff: https://github.com/CoffeaTeam/coffea/compare/refs/tags/v2024.1.0...v2024.2.1
Changelog: https://github.com/CoffeaTeam/coffea/releases/tag/v2024.2.1
Description of changes
Fix build (https://hydra.nixos.org/build/249527501)

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 289012 run on x86_64-linux 1

6 packages built:

python311Packages.coffea
python311Packages.coffea.dist
python311Packages.fsspec-xrootd
python311Packages.fsspec-xrootd.dist
python312Packages.fsspec-xrootd
python312Packages.fsspec-xrootd.dist

I am working on this update in #281235 :)

Duplicate of #281235.


Data Loss - Plus a couple of other issues/bugs

I'm using the latest ArangoDB of the respective release series:

[x] 3.2.3

This one might be related to issue 3226
But I made sure that the keys on both databases are different and don't collide, so better forget about that issue.
-- I did not do a full restart after fixing db keys.
-- I deleted dev db, then created a new empty one, with collection keys starting from 1.
I had a doc on production with key 16315018.
You can see in the video, that page 2 of businesses is not shown caused by this error:
http://localhost:12000/_db/yabco/_api/cursor
Status Code:202 Accepted
REQ:
{"query":"FOR x IN @@collection LET att = APPEND(SLICE(ATTRIBUTES(x), 0, 25), \"_key\", true) LIMIT @offset, @count RETURN KEEP(x, att)","bindVars":{"@collection":"business","offset":10,"count":10}}
http://localhost:12000/_db/yabco/_api/job/150649301542154
Status Code:201 Created
RES:
"warnings":[{"code":1542,"message":"invalid argument type in call to function 'ATTRIBUTES()'"},{"code":1542,"message":"invalid argument type in call to function 'SLICE()'"},{"code":1542,"message":"invalid argument type in call to function 'KEEP()'"}]
and the doc is wiped out!
When I want to recreate it, theres an error because of indexes,
I delete the index,
Try to recreate it, I fail.
Just watch the video:
https://youtu.be/q4pLcAcW0hU
Hopefully I had a backup of the db at 7 am, this disaster happened around 12.

@p30arena is it possible to share the original collection files to investigate the root cause?

@p30arena when you filtering for the _key attribute you have to enclose the _key with double ticks "16315018".

@baslr https://ufile.io/ghb8v
I can give arangodb's core team root access to my vps only if needed.

@p30arena thanks. I will have a look.
Is it possible to share the original database files e.g. the databases directory or the rocksdb directory with which the error occurs?

@baslr
Here it is: https://ufile.io/5lh2o

@p30arena thanks.
I started ArangoDB v3.2.3 and pointing it to your arangodb3 directory. I can access the business collection in the yabco database and filter for and open the document with _key '16315018'. With a probability bordering on certainty the problem is not the arangodb3 directory but maybe a hardware / software problem beyond the scope of ArangoDB.

@baslr
This is my production db, I told before and showed it in the video, I recreated the doc.
Because of these instabilities I take backups every 8 hours, the first link was my backup.
The second one is the data after restoration.
Also I've reported 3 different bugs, I don't know what is confusing you.
1- Second page of the business collection is not shown, You can see the errors in the first msg and in the video.
2- When the data is lost, Indexes are still there, I must delete the indexes so I can recreate the doc with the _key 16315018.
3- I can't recreate the index, because of duplication of data apparently?! but there are no duplicates.

@p30arena so there is no dump of the database directory before deletion of _key '16315018' to have a look at file level what data is there for _key '16315018'?

@baslr Can you find accesses to the doc from the logs?
or can you tell me how to do that?
Unfortunately auditing is only available in enterprise edition.

@p30arena so there is no dump of the database directory before deletion of _key '16315018' to have a look at file level what data is there for _key '16315018'?

no its not available.
I also rechecked our code base, we have no code for deleting a business.

I'm going for a restart,
lets see what happens next.
service arangodb3 stop
Stopping arangodb3 (via systemctl):                        [  OK  ]

it took about 10 seconds.
service arangodb3 start
it took a second.
service arangodb3 status

● arangodb3.service - ArangoDB database server
   Loaded: loaded (/usr/lib/systemd/system/arangodb3.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2017-10-06 18:11:03 GMT; 18s ago
  Process: 6698 ExecStartPre=/bin/chmod 700 /var/lib/arangodb3-apps (code=exited, status=0/SUCCESS)
  Process: 6696 ExecStartPre=/bin/chown -R arangodb:arangodb /var/lib/arangodb3-apps (code=exited, status=0/SUCCESS)
  Process: 6693 ExecStartPre=/bin/chmod 700 /var/lib/arangodb3 (code=exited, status=0/SUCCESS)
  Process: 6691 ExecStartPre=/bin/chown -R arangodb:arangodb /var/lib/arangodb3 (code=exited, status=0/SUCCESS)
  Process: 6688 ExecStartPre=/bin/chmod 700 /var/log/arangodb3 (code=exited, status=0/SUCCESS)
  Process: 6686 ExecStartPre=/bin/chown -R arangodb:arangodb /var/log/arangodb3 (code=exited, status=0/SUCCESS)
  Process: 6684 ExecStartPre=/usr/bin/install -g arangodb -o arangodb -d /var/run/arangodb3 (code=exited, status=0/SUCCESS)
  Process: 6683 ExecStartPre=/usr/bin/install -g arangodb -o arangodb -d /var/tmp/arangodb3 (code=exited, status=0/SUCCESS)
 Main PID: 6700 (arangod)
   Memory: 522.3M
   CGroup: /system.slice/arangodb3.service
           └─6700 /usr/sbin/arangod --uid arangodb --gid arangodb --pid-file /var/run/arangodb3/arangod.pid --temp.path /var/tmp/arangodb3 --log.foreground-tty true

Oct 06 18:11:03 srv39568.screweb.com arangod[6700]: 2017-10-06T18:11:03Z [6700] WARNING {memory} /sys/kernel/mm/transparent_hugepage/enabled is set to 'always'. It is rec...r 'madvise'
Oct 06 18:11:03 srv39568.screweb.com arangod[6700]: 2017-10-06T18:11:03Z [6700] WARNING {memory} /sys/kernel/mm/transparent_hugepage/defrag is set to 'always'. It is reco...r 'madvise'
Oct 06 18:11:03 srv39568.screweb.com arangod[6700]: 2017-10-06T18:11:03Z [6700] WARNING {memory} execute 'sudo bash -c "echo madvise > /sys/kernel/mm/transparent_hugepage/enabled"'
Oct 06 18:11:03 srv39568.screweb.com arangod[6700]: 2017-10-06T18:11:03Z [6700] WARNING {memory} execute 'sudo bash -c "echo madvise > /sys/kernel/mm/transparent_hugepage/defrag"'
Oct 06 18:11:03 srv39568.screweb.com arangod[6700]: 2017-10-06T18:11:03Z [6700] INFO using storage engine mmfiles
Oct 06 18:11:03 srv39568.screweb.com arangod[6700]: 2017-10-06T18:11:03Z [6700] INFO {cluster} Starting up with role SINGLE
Oct 06 18:11:03 srv39568.screweb.com arangod[6700]: 2017-10-06T18:11:03Z [6700] INFO {syscall} file-descriptors (nofiles) hard limit is 131072, soft limit is 131072
Oct 06 18:11:03 srv39568.screweb.com arangod[6700]: 2017-10-06T18:11:03Z [6700] INFO Authentication is turned on (system only), authentication for unix sockets is turned on
Oct 06 18:11:05 srv39568.screweb.com arangod[6700]: 2017-10-06T18:11:05Z [6700] INFO using endpoint 'http+tcp://<IP_ADDRESS>:8529' for non-encrypted requests
Oct 06 18:11:08 srv39568.screweb.com arangod[6700]: 2017-10-06T18:11:08Z [6700] INFO ArangoDB (version 3.2.3 [linux]) is ready for business. Have fun!
Hint: Some lines were ellipsized, use -l to show in full.

The DB Server is breathing fresh air right now.

I'm trying to gather clues.
I can see that DOC 16315018 was somehow corrupted,
I couldn't query it, but when was exploring pages, on 2nd page, I received error:
invalid argument type in call to function 'ATTRIBUTES()
and this is the query:
FOR x IN @@collection LET att = APPEND(SLICE(ATTRIBUTES(x), 0, 25), \"_key\", true) LIMIT @offset, @count RETURN KEEP(x, att)
So what was in x that caused the error?
Then I tried to recreate 16315018, I got an error, it says its there and I can not make  a dup.
What?
I delete the index on _key attr, then I try again, its successful.
So the doc was purged somehow?! but the indexes were still there.
So the query contained a doc that was pointing to some VOID space.
I can show you a more interesting Video that shows another BUG:
https://youtu.be/RNS7700XlWU

@p30arena your advertising bug is not a bug.

You could switch to https://en.wikipedia.org/wiki/UBlock_Origin.

@baslr  haha It was due to AdBlocker, what a nasty extension 😸

@p30arena
regarding your data loss issue here, was it a fresh arangod installation or does the arangod experienced a double restore as in #3226?

@baslr It was a fresh installation, then an arangorestore...

@baslr Do you suggest I clear the whole data, and restore it again?

@p30arena
yes, do a clean reinitialization of the database.
If this problem occurs again, please do a copy of the database directory before everything else so we can investigate the database files.

@p30arena I close this issue now. If the error appears again please reopen the issue and we will happy to investigate.


Gitpod/randi.w.2012/2022 10 24 12 09 28

@randi-c-dubs Thank you! just making this PR to keep track of things
(I'm trying to get a github action working to automatically make PRs from gitpod branches, but I've not yet had luck to get it to actually make the PR)


For hacktoberfest  adding some new animation

@SamarthTMSL  pls assign this ticket, so I can add some cool animation.


Revert popover-polyfill back to v0.3.8

What are you trying to accomplish?

Unfortunately v0.4.0 of popover-polyfill references window, which breaks React SSR in github dotcom.
Integration

No changes necessary in production.
Risk Assessment

[x] Low risk the change is small, highly observable, and easily rolled back.
[ ] Medium risk changes that are isolated, reduced in scope or could impact few users. The change will not impact library availability.
[ ] High risk changes are those that could impact customers and SLOs, low or no test coverage, low observability, or slow to rollback.

Accessibility

No new axe scan violation - This change does not introduce any new axe scan violations.

I was able to add a shim in dotcom so popover-polyfill is not executed in SSR mode 🎉


Update clamav-scan task image

Update the image used by the clamav task. This may fix the timeout issue we were seeing.

This is fixed in #512.


Small fixes and additions to the docs

Summary of the Pull Request
Some small changes in the documentation:

A typo in the in the overview page.
Adds a step to the installation instructions to stop Docker Desktop (when running).
Some small textual changes in the Faq.

Pull Request Checklist

[x] Documentation updated.

Thanks Edwin for reading the docs :-).


ele element is missed

Please ignore this

You can close if it's ok for you :)

Please ignore this

The ele element also seems to be missed for me in a Trkseg's Wpt, what did you do to get the elevation?


Refactor Front-end Unit Test Suite

Expected Behavior

Front-end should have a run-able unit test suite so that we can validate new component/works.
Current Behavior

The current unit test suite does not run and it is using deprecated packages. It also does not setup to run on docker.
Possible Solution

Adding docker build
Upgrade test tool dependency or switch to better tool.

Steps to Reproduce

There is no way to run front-end test at the moment.

Context (Environment)

Detailed Description

Possible Implementation

Checklist before submitting

[x] I have confirmed this using the officially supported Docker Compose setup using the local.yml configuration and ensured that I built the containers again and they reflect the most recent version of the project at the HEAD commit on the master branch
[x] I have searched through the other currently open issues and am confident this is not a duplicate of an existing bug
[x] I provided a minimal code snippet or list of steps that reproduces the bug.
[x] I provided screenshots where appropriate
[x] I filled out all the relevant sections of this template

Work on this has already been started and will be published in #239

Fixed in https://github.com/concept-to-clinic/concept-to-clinic/commit/09d4bc5863f2fcf7e166d4b15cef01779a8b8dd4


Model accurate missing after converting to tflite

1. System information

Windows 10
Pip package
2.7.0

2. Code
import tensorflow as tf
model = tf.keras.models.load_model('liveness.model')
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()
open("liveness4.tflite", "wb").write(tflite_model)
3. Failure after conversion
If the conversion is successful, but the generated model is wrong, then state what is wrong:

Model produces wrong results and/or has lesser accuracy.

Also Im  getting this warning after converting to tflite model.
WARNING:absl:Buffer deduplication procedure will be skipped when flatbuffer library is not properly loaded

@Ajith61
In order to expedite the trouble-shooting process here,Could you please fill the issue template,
Thanks!


windows clipboard not imported

goneovim 0.4.1 (+ neovim 0.4.3) cannot use the content from Windows clipboard.
it can export.
neovim alone works both ways with following settings
set clipboard=unnamed "to use system clipboard. win32yank needed on Win
set clipboard+=unnamedplus

plugin miniyank was the cause


75 rdm contacts page

Hi Nami, here it the page, please let me know what do you think.
I also added the "lock" when the page only lives inside myEUR

I'm taking over to update the remaining changes—hope it's ok 🙏


Fix PySide2 issues on QActionEvent

Closes #63.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

1 of 4 (25.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 40.485%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pyqtribbon/menu.py
1
4
25.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
534

Relevant Lines:
1319

💛  - Coveralls

Errors
AttributeError: PySide2.QtGui.QActionEvent has no attribute action()
TypeError: unhashable type: 'RibbonContextCategories'


Use chocolatey and version bump

Use the chocolatey/chocolatey module.

Have you validated that the chocolatey/chocolatey module is stable enough for regular use?

It's the same, he just moved it out of his personal account.


Core strict compiler option

Short description of what this resolves:
This turns on the strict compiler option in packages/core/tsconfig.json and merges the current core branch
Ionic Version: 1.x / 2.x / 3.x
4.x

Thanks for the PR! We are going to be merging the core branch into master soon. If you'd like to make a PR just for the tsconfig file we can review it. Thanks! 🙂


Consider normalizing submitted flags before checking

The current code uses substring (python: submitted_flag in desired_flag). This leads to surprises when there are, say, leading/trailing spaces in the submitted flag.
P.S. For competition, most problems have the flags in a special format but there are some exceptions. For the problems in the latter group (no format), this can create confusion if some user formats the flag because now the flag contains extra characters.

After brief discussion with problem devs, what would work better would be a case-insensitive flag on the problem, carried over from the problem.json config.


By default, vb razor views are not used for reference detection.

I downloaded this tool to find orphan resource string but it ended up giving me half of my string being orphan. After looking around, I found out my .vbhtml files were not being checked.
All I had to do was change the Detect Code Reference configuration by adding the .vbhtml file extension to a check:

It now works fine but I think it would be safe and better to have that by default. There are already rules for vb files. Why not add .vbhtml files.

@IamCloud Looks you're the first one using this. Isn't  .vbhtml case insensitive?

@IamCloud Looks you're the first one using this. Isn't  .vbhtml case insensitive?

Yes, I believe you are right. Maybe add another record with the same regex but Case Sens. unchecked.


Script to invoke the best tip from neighbors

Done but needs a deployment method.  @yourbuddyconner  is working on that.

@yourbuddyconner  - working on a way to deploy in the test net.  Any update?

I took some time today to slap together the rest of the script. It can be deployed locally for adhoc testing, I am going to include it in a future commit to coda-automation but here it is for posterity: https://gist.github.com/yourbuddyconner/ea9aeb5729722cc1a1d4ba2a5ac27b31

The script(s) were included in the coda-automation commit cf53839b4d59da9fdd2c6884b34197faf63936c9


Data deanonymization

Description
The feature for pseudonymizing data with ability to retrieve original text (deanonymization) has been implemented. In order to protect private data, such as when querying external APIs (OpenAI), it is worth pseudonymizing sensitive data to maintain full privacy. But then, after the model response, it would be good to have the data in the original form.
I implemented the PresidioReversibleAnonymizer, which consists of two parts:

anonymization - it works the same way as PresidioAnonymizer, plus the object itself stores a mapping of made-up values to original ones, for example:

    {
        "PERSON": {
            "<anonymized>": "<original>",
            "John Doe": "Slim Shady"
        },
        "PHONE_NUMBER": {
            "111-111-1111": "555-555-5555"
        }
        ...
    }

deanonymization - using the mapping described above, it matches fake data with original data and then substitutes it.

Between anonymization and deanonymization user can perform different operations, for example, passing the output to LLM.
Future works

instance anonymization - at this point, each occurrence of PII is treated as a separate entity and separately anonymized. Therefore, two occurrences of the name John Doe in the text will be changed to two different names. It is therefore worth introducing support for full instance detection, so that repeated occurrences are treated as a single object.
better matching and substitution of fake values for real ones - currently the strategy is based on matching full strings and then substituting them. Due to the indeterminism of language models, it may happen that the value in the answer is slightly changed (e.g. John Doe -> John or Main St, New York -> New York) and such a substitution is then no longer possible. Therefore, it is worth adjusting the matching for your needs.
Q&A with anonymization - when I'm done writing all the functionality, I thought it would be a cool resource in documentation to write a notebook about retrieval from documents using anonymization. An iterative process, adding new recognizers to fit the data, lessons learned and what to look out for

Twitter handle
@deepsense_ai / @MaksOpp

per slack:
only nit: in this example it would be good to set temperature=0 (its weird that the first, not de-anonymized response is different than the second) and maybe make it so the llm response isnt EXACTLY the same as the input (otherwise its unclear if anything is happeneing

@hwchase17 Done! I came up with another example, let me know what you think :smile:

@hwchase17 checks have passed, ready to merge :+1:


app_uninstalled received after installing

Description
Its been two times that I end up stuck in a install / uninstall loop
What type of issue is this? (place an x in one of the [ ])

[x] bug

Bug Report
Filling out the following details about bugs will help us solve your issue sooner.
Steps to reproduce:
I didnt managed to reproduce it on purpose, I think its when the app is failing to get the app_uninstalled event, slack retry to send it later. When the app have been already reinstalled, so the event is lying.
I uninstall the app, then I get this error when the uninstall event arrive
WARN 2021-09-01T12:43:13.114Z : Authorization of incoming event did not succeed. No listeners will be called

Then I reinstall and seconds or 1 minute after I receive a new  lying app_uninstalled event which cause problem.
Expected result:
Should not receive app_uninstalled when the app is installed.

Hello @adelin-b, this part of your question is key:

I think its when the app is failing to get the app_uninstalled event, slack retry to send it later.

As described in our Events Guide on Responding to Events:

Your app should respond to the event request with an HTTP 2xx within three seconds. If it does not, we'll consider the event delivery attempt failed. After a failure, we'll retry three times, backing off exponentially.

After any kind of failure (whether a failure to respond from your bolt app, or a different failure), Slack will retry to send the event up to three times. As such, indeed, in fast uninstall/re-install of the app scenarios, if your app fails to respond to the event for whatever reason, the uninstalled event may be re-delivered after an exponential backoff.
I am not sure the bolt framework can do anything here to mitigate this. For event-based APIs it is a common pattern to retry failed deliveries with an exponential backoff. I think the key to resolving this issue is identifying why your app is erroring or failing to respond to the initial event. I understand that in this uninstall/reinstall scenario, multiple deliveries of these events may be confusing and hard to deal with, but multiple deliveries of the events would only happen in the case that the original deliveries are not responded to appropriately.
Do you have further logging in the app as to why authorizing the incoming event did not succeed? The warning you posted comes from a particular section of bolt-js which also invokes an error handler. This error handler, by default, will log out the details of the authorization error. Do you see any further details in your app's logs around why this was caused?

Hi @adelin-b - Any luck with troubleshooting the cause of the authorization error via the logs? Let us know!

Solved the issue by attempting a request and making sure it get in error before removing the data associated with the current install.

It's great to know you've resolved the issue! Let us close this issue now.


not working

databag dhcp_networks is ignored

Could I get more information on this issue please?

@jmccann sorry, this is from a deleted user. May as well close.

HAH!  Didn't even notice ... thanks!