⚠ Bofrost.at has degraded performance

In aa2b707, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 2796 ms

Resolved: Bofrost.at performance has improved in b6e3b38 after 8 minutes.


Mandatory omit a field if some other field equals some value

[X] I have looked at the documentation here first?
[X] I have looked at the examples provided that may showcase my question here?

Package version eg. v9, v10:
v10.4.0
Issue, Question or Enhancement:
Mandatory omit a field if some other field is present in the struct already.
Code sample, to showcase or reproduce:
For example-
type A struct{
    B string
    C string
}

C should definitely not be there if B equals something like "abcd"

The excluded_if tag is needed, as opposed to required_if.

Hello, is excluded_if implemented at all? I am unable to find it in the documentation nor in the source.


Update sql-server-ag-kubernetes.md

First pull request.

@echostark9 : Thanks for your contribution! The author, @MikeRayMSFT, has been notified to review your proposed change.

Thank you for the correction!

Reviewed editorial correction.


Fix typo on yarn deploy --network code example

Small typo on "--nework" instead of "--network" but makes the command fail for those that copy it 😅

Lol yup, Tysm 🙌


OkLCH sliders?

Hi,
I think it would be great to have a set of OkLCH sliders.
In general Chroma would indicate better relative color strengths among colors than Saturation.
Best,
Alex

Hi,
Thanks for the suggestion. The sliders available are what comes by default with Godot colour pickers. It would be nice to have another option, but the R&D involved is out of my reach right now. I'll keep it in mind though once Godot adds more things to their pickers.
For the results, the interpolation used is Oklab which as far as I am aware preserves the relative colour lightness. I think it's worth testing if allowing blend between this and OkLCH for each colour strip produces better results.
Cheers,
V

I think the current interpolation in OkLab is quite great. Somehow the palette feels more organic than OkLCH interp, which tends to force the path to be an arc.
But I'd suggest an option to make the curve bend in the third dimension. I mean right now the curves lies flat on a plane only. Maybe the introduction of a fourth point/value would permit such geometry to happen.
Thanks so much for the reply and the great tool!
Cheers,
Alex


[LANG-1228] Prefer Throwable.getCause() in ExceptionUtils.getCause()

For convenience: https://issues.apache.org/jira/browse/LANG-1228

+1

Thanks! :+1:


Remove the Underline from the Hello, Sign In option in the Header

Remove the Underline from the Hello, Sign In option in the Header.

Hey @Jai-Solania-29 can I fix these issue.

Yes, please!!


Can't get user info via user_info2 anymore

Please follow the guide below

Issues submitted without this template format will be ignored.
Please read the questions carefully and answer completely.
Do not post screenshots of error messages or code.
Put an x into all the boxes [ ] relevant to your issue (==> [x] NO spaces).
Use the Preview tab to see how your issue will actually look like.
Issues about reverse engineering is out of scope and will be closed without response.
Any mention of spam-like actions or spam-related tools/libs/etc is strictly not allowed.

Before submitting an issue, make sure you have:

[x] Updated to the lastest version v1.6.0
[x] Read the README and docs
[x] Searched the bugtracker for similar issues including closed ones
[x] Reviewed the sample code in tests and examples

Which client are you using?

[ ] app (instagram_private_api/)
[x] web (instagram_web_api/)

Describe the Bug/Error:
When I am trying to fetch user_info via user_info2 method in web_api, I am getting error. It works fine in the initial request, but not in the following requests. Here is the code and the response I get. I also faced the issue when I try in the browser as well. I think Instagram has updated its API. Anyone encountering this bug?
for (;;);{"__ar":1,"error":1357004,"errorSummary":"Sorry, something went wrong","errorDescription":"Please try closing and re-opening your browser window.","payload":null,"hsrp":{"hblp":{"consistency":{"rev":1005632913}}},"lid":"7105412861952089317"}

Python 3.6.10 :: Anaconda, Inc.
Code:
api = Client(settings=cached_settings)
user = api.user_info2('instagram')

Error/Debug Log:
JSONDecodeError                           Traceback (most recent call last)
<ipython-input-67-d47f0623923e> in <module>
----> 1 user = api.user_info2('instagram')
      2 user

~/anaconda3/envs/myenv/lib/python3.6/site-packages/instagram_web_api/client.py in user_info2(self, user_name, **kwargs)
    431        endpoint = 'https://www.instagram.com/{username!s}/'.format(**{'username': user_name})
    432        try:
    433             info = self._make_request(endpoint, query={'__a': '1'})

~/anaconda3/envs/myenv/lib/python3.6/site-packages/instagram_web_api/client.py in _make_request(self, url, params, headers, query, return_response, get_method)
    283             self.logger.debug('RES BODY: {0!s}'.format(response_content))
--> 284             return json.loads(response_content)
    285
    286         except compat_urllib_error.HTTPError as e:

~/anaconda3/envs/myenv/lib/python3.6/json/__init__.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    352             parse_int is None and parse_float is None and
    353             parse_constant is None and object_pairs_hook is None and not kw):
--> 354         return _default_decoder.decode(s)
    355     if cls is None:
    356         cls = JSONDecoder

~/anaconda3/envs/myenv/lib/python3.6/json/decoder.py in decode(self, s, _w)
    337
    338         """
--> 339         obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    340         end = _w(s, end).end()
    341         if end != len(s):

~/anaconda3/envs/myenv/lib/python3.6/json/decoder.py in raw_decode(self, s, idx)
    355             obj, end = self.scan_once(s, idx)
    356         except StopIteration as err:
--> 357             raise JSONDecodeError("Expecting value", s, err.value) from None
    358         return obj, end

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

same here

https://stackoverflow.com/questions/11796349/instagram-how-to-get-my-user-id-from-username/38342137#38342137


Use argument unpacking in Facade abstract class

Replace switch and call_user_func_array with argument unpacking.
Argument unpacking (...$var) is available since PHP 5.6.

concrete5 8.x should be compatible with PHP 5.5, isn't it?

Could be, I'm not sure. @aembler it'd be nice to put the requirements in https://github.com/concrete5/concrete5/blob/develop/CONTRIBUTING.md, don't you think?

It's here: https://github.com/concrete5/concrete5/blob/develop/composer.json#L15

And here: https://github.com/concrete5/concrete5/blob/develop/concrete/controllers/install.php#L435

Ok, let's close this one then. Pity ;) Would've been a nice change.


update Earth discipline structure

fixes #66
fixes #38
fixes #34

Notes:

reorganized levels can be reviewed in context at https://github.com/wmo-im/wis2-topic-hierarchy/tree/test-structure/topic-hierarchy (this PR branch)
core levels are relational, and stay as 1 CSV per level
Earth discipline levels are hierarchical and organized in a directory tree.  Sublevels are put forth as directories.  Leaf levels are put forth as index.csv files
CI updated as follows:

on each commit to main branch generates https://wmo-im.github.io/wis2-topic-hierarchy/wth-bundle.zip which are all core level CSVs and a bundled CSV of the Earth system discipline.  The zipfile is organized in this manner given the abovementioned

TT-WISMD 2023-11-17:

removed earth-system-discipline-subcategory from topic-hierarchy.csv
add text to specification

domains SHALL use a hierarchy only (not relational)
experimental SHOULD NOT be a permanent solution

@wmo-im/tt-wismd FYI this PR has now been updated per the discussion in https://github.com/wmo-im/wis2-topic-hierarchy/pull/70#issuecomment-1816459037.  Please review and provide feedback or approve.


ABTesting Swift still uses FIR prefix

This should be removed using the NS_SWIFT_NAME macro.

This change was submitted internally and will be released along with the Firebase 6 milestone.


Help on applying local forces

Hi,
I'm trying to build a space racer. The ship is a rigidBody and I try to set a local force (for example) at the position of the main engine when space is pressed.
I first thought, that I have to set the direction of the force in the local coordinate system of the object. For example
position = (0, 0, -shipLength/2) and direction (0, 0, -force).
But this just led to spinning of the ship. I then tried to multiply the direction by the ships orientation quaternion. This works better (so it seems like the direction vector is meant to be in world coordinates). But it's still not flying on a straight line.
Some code:
When the ship is initialized...
_ship.addThruster(
    ThrusterEnum::main,                // main engine
    vec3(0, 0, SHIP_BODY_LENGTH / 2),  // pos of thruster
    vec3(0, 0, 1),                     // direction of thruster
    100000                             // thrust
);

And in the update method of the ship...
for (uint i = 0; i < (uint)ThrusterEnum::NUM; ++i) {
    const auto& thruster = thrusters[i];
    const rpQuat rpQuat = body->getTransform().getOrientation();
    const rpVec3 localForce = toRp3d(thruster.getCurrentForce());   // direction of the thruster, magnitude depending on user input
    const rpVec3 forceDirectionInWorldCoordinates = rpQuat * localForce;
    const rpVec3 rpPos = toRp3d(thruster.getPosition());

    if (localForce.length() > 0) {
        body->applyForceAtLocalPosition(
	    forceDirectionInWorldCoordinates,
	    rpPos
        );
    }
}

Any idea what I do wrong? What else can I add here?
Thanks for any tip!!
And thanks for the nice library!!

Hello,
There are three different methods to apply a force to a body.
RigidBody::applyForceAtLocalPosition(const Vector3& force, const Vector3& point)
Here the force needs to be in world-space coordinates (I need to add this in the documentation), but the point where to apply the force needs to be in local-space coordinates.
RigidBody::applyForceAtWorldPosition(const Vector3& force, const Vector3& point)
Here the force needs to be in world-space coordinates, and the point where to apply the force needs to be in world-space coordinates too.
RigidBody::applyForceToCenterOfMass(const Vector3& force)
Here the force needs to be in world-space coordinates and the force is applied to the center of mass of the body.
Concerning the rotation of the body when you apply a force. Note that if you apply a force to a body at a location that is different from the center of mass, some torque is created and the body will start rotating. If you do not want the body to start rotating, you need to apply your force to the center of mass for instance by using the RigidBody::applyForceToCenterOfMass() method above.
I hope this helps.

So it seems like my understanding is roughly correct so far, thanks a lot for helping me verifying this!
The problem is, I guess there should be no rotation with the current values (only z direction), as the center of mass and the thruster should be on a line parallel to the direction of the force. (The documentation states, that the center of mass is in the local origin on default):
local pos => (0, 0, SHIP_BODY_LENGTH / 2)
local direction => (0, 0, 1)

And then when applying the local force:
const rpQuat rpQuat = body->getTransform().getOrientation();
const rpVec3 forceDirectionInWorldCoordinates = rpQuat * localForce;

body->applyForceAtLocalPosition(
  forceDirectionInWorldCoordinates,
  rpPos
);

Do you have a best guess with your experience? Maybe rounding errors?!

As long as you do not call RigidBody::updateMassPropertiesFromColliders() or RigidBody::setLocalCenterOfMass() methods the center of mass of your body is at (0, 0, 0) in local-space. You can use the RigidBody::getLocalCenterOfMass() to make sure of this. In this case, if your local force is aligned with the z-axis, there should be no torque when applying your force at (0, 0, length/2) in local-space.
As you said maybe, rounding errors during the computation of the world-space force make the force not perfectly aligned with the z-axis.
Could you try to use the RigidBody::applyForceToCenterOfMass() instead? If you do not want any torque, you should call this method. Let me know if this works.

Do you have any news about this? Can I close this issue?


Update readme for controller local debugging

@richardwxn: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

e2e_operator
23e844ec6503ec427b047a85d676c4f52cf2f227
link
/test e2e_operator

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


Tracker issue towards 1.0 -- API stability

NIFTI-rs is already two and a half years old. Despite not having many known dependents, I think it's at least time to start thinking about what we'd like to have in 1.0. The principle of this version is that it should transmit a stronger signal of API stability and a sufficient level of completeness. New features can be introduced later on, of course, but if those require significant public API changes, it would be better to push them before 1.0.
The list below is currently a draft. Please let me know of what should be taken care of in this initiative towards crate stability.

[ ] Volume access API: will the current signature of NiftiVolume satisfy future implementations?
[ ] Object/Volume writing API: currently only available in combination with ndarray, and is also not uniform across data types (RGB); is a uniform volume saving method feasible?
[x] #60 AsPrimitive ergonomics: most volume manipulation methods infect other functions in order to check whether the data element type is OK. Making this better is likely to require a breaking change of the public API.
[ ] NiftiHeader design:  should this stay as a repr(Rust) struct with public fields? Should descrip become an 80-element array to avoid a heap allocation?
[ ] Dependencies in public API: I can count at least the following crates that are part of this library's public API: ndarray, nalgebra, simba, andbyteordered. It is conventional to only release a 1.0 if the public API dependencies are not pre-1.0, since that could disrupt the intended signal of stability.

This is somewhat difficult for me to say because I'm only aware of my use-cases. Imeka is a heavy user of nifti-rs: we read and write thousands of images, of almost all types, 3D and 4D. We use it mostly for

reading only the header to access/modify some information, like the affine
reading data to ndarray
writing back data from ndarray to nifti

We never used the volume access API and we don't plan to use it in its current form, so we have no intention to use the volume writing API. ndarray is just too convenient.
In brief, we are quite happy with nifti-rs because it offers what we need and more. My main "problem" with nifti-rs is currently #60 because it makes my code uglier than it should be :) Imo, the rest is mostly nice-to-have, but absolutely not a blocker for us.

Thanks for the feedback.

reading only the header to access/modify some information, like the affine

This reminds me that one of the things I do not particularly like in the current header definition is the vector allocation for the description. While const generic functionality is quickly entering into the compiler, we still don't know when it'll be done, and it still seems unreasonable to wait for it to become stable. Before it happens, I could probably devise a custom 80-byte array type that would become part of the public API without becoming a hindrance in the future.

We never used the volume access API and we don't plan to use it in its current form, so we have no intention to use the volume writing API. ndarray is just too convenient.

It truly is, and I agree to continue supporting the means to convert any volume implementation into an ndarray. On the other hand, from the beginning I have pictured the possibility of accessing volumes which are not entirely in memory, thus why I chose not to throw it away. I would like to be sure that one can make a generic implementation of save, which would work with or without the volume being backed by an ndarray.

My main "problem" with nifti-rs is currently #60 because it makes my code uglier than it should be :)

Yep, I also think this is worth improving, and if we don't do it before 1.0, it likely won't happen until a hypothetical 2.0. I'll put it on the list.

Hi
first of all thank you for your work on this project!
I wanted to ask in general about the state of this project as development seems to have stopped for a while and especially nifti 2 support (#96) and writing volumes seem like big missing features.
I'd be interested in helping with the development.
My side goal would be to write a wrapper for python/numpy.

Hi @nx10. Development has stopped because nifti-rs answers all our current needs. The two persons that wanted to contribute the Nifti2 feature suddenly stopped answering. As we don't need that feature in our projects, we left it like that.
Of course, we would gladly accept their contribution, and yours. I think we are quite responsive (@Enet4 and I) and halpful if you have some questions or a MR.

My side goal would be to write a wrapper for python/numpy.

Hum, what do you mean? Isn't NiBabel sufficient for your need? Is it too slow?

@nilgoyette thanks for the fast response! Yes exactly, nibabel is quite slow. Although it is hard to say how much faster nifti-rs would be without benchmarking it. I've done some tests with using rust to instantiate numpy arrays from freesurfer files (which are trivial / basically just arrays) and seen a ~3x speed increase. With nifti being a more complex format I became interested in wrapping this library.
In general I think it would be great to have a number of wrappers for e.g. Python, R, Node.js that all use nifti-rs as a native/unified "backend".

My experience is that NiBabel is indeed slow, at least for the loading and saving parts. Once the data is in NumPy, it's almost as fast as C. I created trk-io because NiBabel was too slow and we're now 15-20x faster. I wouldn't be surprised if nifti-rs is that much faster than NiBabel.

Hi @Enet4 ! Thank you for your reply and explanation as well!
Yeah, I scrolled through the PR and did wonder why they decided to merge the implementations. Seems like that should be handled by a higher level abstraction.
You being open to provide basic maintenance is good enough for me to start work on (Python) bindings. Once I got around to build a basic prototype I will open a new issue so we can discuss further.
Thanks!


TranslationServer.translate() non-functional in editor

Tested versions
v4.3.stable.nixpkgs [77dcf97d8]
System information
nixos-unstable
Issue description
TranslationServer seems to not work in the editor at all, witch I partly understand, but especially it not working when being explicitly called from script is a hindrance to me right now.
Steps to reproduce

check the console after opening main scene
run the game

EXPECTED:
prints "Es ist Mittwoch, meine Kerle!" both times.
OBSERVED:
only prints the translated message in game.
Minimal reproduction project (MRP)
i18-editor-bug.zip

This is expected currently.
TranslationServer works, but project translations are not loaded in the editor, and translations are disabled for the edited scene.
Just curious, what's your use case for TranslationServer.translate()? If you do need to call that in some tool script, you can load your translations manually and add them via TranslationServer.add_translation().

I have a tool that dynamically stitches together a text dynamically based on translated sub-stings. I use that to gradually reveal text as it is spoken. And because the voice lines and text can have different languages, that was the most practical thing to do.
Also: German and English have quite different lengths, so it is really difficult to work on ui when I cannot test with different translations in editor, even when I explicitly force it in code.
Also: if no locale is loaded in the editor, then
TranslationServer.get_loaded_locales()
should not return anything in editor.
I don't think your workaround works btw:
print(TranslationServer.get_loaded_locales())
TranslationServer.add_translation(preload("res://translation.de.translation"))
print(TranslationServer.translate("wensday"))

prints
["de", "en"]
wensday

I don't think your workaround works btw

Sorry, I confused the changes I'm making with the stable version / master. I am working on removing the special handling for editor translations, so that the editor's translation workflow is consistent with the game's translation workflow, so that translations can be previewed in the editor and the TranslationServer is works regularly. That could solve this issue I think (that PR still needs some work though).
For your use case, I think you can use preload("res://translation.de.translation").get_message("wesday") directly (the difference is that it returns an empty string when there is no translation for the text.


Proposal to update pull request workflow

**Please describe what should be documented
This issue is to discuss a proposal to change the way pull requests should be created are merged.
Per the new proposal, contributors are expected to do the following:

Fork the osm repo
Get your local branch up to date with upstream's main branch (git pull --rebase upstream main or git fetch upstream && git rebase upstream/main). Please do not use git pull as it does a merge that makes the commit history messy.
Push changes to fork and open a pull request against the main branch from fork
Get code reviewed
After review approvals, rebase to main branch as described in step 2 (git pull --rebase upstream main or git fetch upstream && git rebase upstream/main) and squash commits in the PR to a single commit that follows the commit message guidelines (signoff etc.)
Force push your changes (git push -f)
Once your PR is approved and commits are squashed, a maintainer will merge your change using the merge pull request option as a merge commit to main.

**Please suggest where in the repo the document should be located
https://github.com/openservicemesh/osm/blob/main/CONTRIBUTING.md

Maybe we should ask folks to have their commits squashed and rebased before the review process? Then, we could just require folks to rebase if there is a notification of merge conflicts with main.

Maybe we should ask folks to have their commits squashed and rebased before the review process? Then, we could just require folks to rebase if there is a notification of merge conflicts with main.

Maybe we should also ask folks to squash commits before the review phase in step 2?

Maybe we should also ask folks to squash commits before the review phase in step 2?

Maybe we should also ask folks to squash commits before the review phase in step 2?

I am not against that, that's the way I create my PRs. For authors, we can incorporate this as a recommendation. Is there a reason this must happen at pull request creation time?

Maybe we should also ask folks to squash commits before the review phase in step 2?

I am not against that, that's the way I create my PRs. For authors, we can incorporate this as a recommendation. Is there a reason this must happen at pull request creation time?

I am not against that, that's the way I create my PRs. For authors, we can incorporate this as a recommendation. Is there a reason this must happen at pull request creation time?

👍  Just so there is less churn between review approval and PR merge.

I am not against that, that's the way I create my PRs. For authors, we can incorporate this as a recommendation. Is there a reason this must happen at pull request creation time?

👍  Just so there is less churn between review approval and PR merge.


Pro 1.2 Unable to create METOC editing templates

The METOC editing templates seem to be missing.
Pro Version: <IP_ADDRESS>52
Priority (1.2): No
Repro Steps:

Open the pro project found here: https://github.com/Esri/military-features-data/blob/master/data/mil2525d/sample_data/projects/MilitaryFeatures.ppkx

Under the "Edit" tab, select the "Create Feature" icon. You'll see that there are missing feature templates:

Click to see the templates not listed, you'll find the METOC points, lines, and areas:

If you go to the Manage Templates pane, and select METOC points, lines or areas, you'll find that the "Layer has no templates."

I was unable to create the templates (Pro crashed)

@jeconley could you enter a CR for this issue.

@Dbarnes1 Can you please review the domains and subtypes for the METOC layers, specifically for the METOC Space ones?  These seem to be missing values.  Phil on the editing team has taken a look and is not sure what the issue is but saw that the domains and subtypes look suspect.  The hope is that adding these values will fix the issue.
For the Control Measure layers, these were just set to not be editable.  If you make them editable then the feature templates will be added.

The issue of the attributes missing from the attribute pane and template pane has been fixed and should be available in the daily build.  This doesn't fix the issue of the templates not being created but will allow you to create them manually.  The hope is that populating the domain will fix the issue of the templates not being created automatically.
From Phil Sanchez:
"John installed a fix yesterday that resolves the missing attributes for the METOC layers. I was able to create a new feature template (manually) and then change the subtype to something valid (via the Symbol Set field), and create new features in the map. You will need to advance to the Active Template pane view to set attributes prior to drawing a feature.
This doesn’t resolve the missing feature templates issue with these layers, but does allow users to create their own templates. Hopefully it’s just a matter of populating the domain with the proper values that would generate the symbols."

METOC editing templates have been successfully created manually:

thanks @jeconley !


Check .vimrc for possible misconfiguration

Fixes #1489
RELEASE_NOTES=[ENHANCEMENT] Warn if vim might be leaking secrets
Signed-off-by: Dominik Schulz<EMAIL_ADDRESS>

Codecov Report

Merging #1494 into master will decrease coverage by 0.11%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master    #1494      +/-   ##
==========================================
- Coverage   61.03%   60.92%   -0.11%
==========================================
  Files         219      219
  Lines       12211    12233      +22
==========================================
  Hits         7452     7452
- Misses       3936     3957      +21
- Partials      823      824       +1

Impacted Files
Coverage Δ

internal/action/edit.go
51.67% <0.00%> (-1.78%)
:arrow_down:

internal/editor/editor.go
32.73% <0.00%> (-18.70%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 212da3a...52321f6. Read the comment docs.


Withdraw unstaked tokens

Resolves #153
Resolves #150
video.webm
video.webm
video.webm
video.webm
video.webm

@race-of-sloths include

@Megha-Dev-19 Thank you for your contribution! Your pull request is now a part of the Race of Sloths!
Congratulatory messages are spent out! Your results have exceeded our imagination!

Current status: waiting for scoring
We're waiting for maintainer to score this pull request with @race-of-sloths score [0,1,2,3,5,8,13] command. Alternatively, autoscoring [1,2] will be applied for this pull request
What is the Race of Sloths

Race of Sloths is a friendly competition where you can participate in challenges and compete with other open-source contributors within your normal workflow
For contributors:

Tag @race-of-sloths inside your pull requests
Wait for the maintainer to review and score your pull request
Check out your position in the Leaderboard
Keep weekly and monthly streaks to reach higher positions
Boast your contributions with a dynamic picture of your Profile

For maintainers:

Score pull requests that participate in the Race of Sloths and receive a reward
Engage contributors with fair scoring and fast responses so they keep their streaks
Promote the Race to the point where the Race starts promoting you
Grow the community of your contributors

Feel free to check our website for additional details!

Bot commands

For contributors

Include a PR: @race-of-sloths include to enter the Race with your PR

For maintainers:

Invite contributor @race-of-sloths invite to invite the contributor to participate in a race or include it, if it's already a runner.
Assign points: @race-of-sloths score [1/2/3/5/8/13] to award points based on your assessment.
Reject this PR: @race-of-sloths exclude to send this PR back to the drawing board.
Exclude repo: @race-of-sloths pause to stop bot activity in this repo until @race-of-sloths unpause command is called

@race-of-sloths score 8


Login to CM2000 - Failing :: loginUrl = loginForm[0].get("action")

Doesn't look like the login function can reliability pull the cookie from the CM200.
INFO:app:Logging into modem
[] <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< Added :: print(loginPage.select("#target"))
Traceback (most recent call last):
File "/Volumes/WinBucket/files/script-dev/CableModem/retriever.py", line 22, in wrapper
return job_func(*args, **kwargs)
File "/Volumes/WinBucket/files/script-dev/CableModem/retriever.py", line 42, in collectionJob
modem.login()
File "/Volumes/WinBucket/files/script-dev/CableModem/modems/netgear_cm2000.py", line 148, in login
loginUrl = loginForm[0].get("action")
IndexError: list index out of range
~~
Tried to loop through the login process until it is != [] .. Need to debug some more to see if I can figure out what is going on ..
while loginPage.select("#target") == []:
# response = []
time.sleep(10)
response = self.session.get(self.baseUrl, verify=False)
loginPage = BeautifulSoup(response.content, features="lxml")
print(loginPage.select("#target"))
~~
--|--

Cable Firmware Version
V8.01.02

Might me a timing issue while testing in somewhat rapid succession while testing data collection on my side ..
I am catching the  loginPage.select("#target") == []: and triggering a second auth attempt that gets me the cookie ..
def login(self):
    self.logger.info("Logging into modem")

    modemAuthentication = {
        'loginName': "admin",
        'loginPassword': self.config['Modem']['Password']
    }

    response = self.session.get(self.baseUrl, verify=False)

    # Get login url
    loginPage = BeautifulSoup(response.content, features="lxml")

    print("Before Loop: ", loginPage.select("#target"))

    while loginPage.select("#target") == []:
        response = []
        time.sleep(2)
        response = self.session.get(self.baseUrl, verify=False)
        loginPage = BeautifulSoup(response.content, features="lxml")
        print(loginPage.select("#target"))

    loginForm = loginPage.select("#target")
    loginUrl = loginForm[0].get("action")

    response = self.session.post(self.baseUrl + loginUrl, data=modemAuthentication, verify=False)

I have been seeing the error inconsistently and a different error.
Traceback (most recent call last):
File "/root/cablemodem-status/retriever.py", line 22, in wrapper
return job_func(*args, **kwargs)
File "/root/cablemodem-status/retriever.py", line 42, in collectionJob
modem.login()
File "/root/cablemodem-status/modems/netgear_cm2000.py", line 136, in login
loginUrl = loginForm[0].get("action")
IndexError: list index out of range
INFO:app:Running as daemon
^CTraceback (most recent call last):
File "/root/cablemodem-status/retriever.py", line 112, in
time.sleep(1)
KeyboardInterrupt
root@CasaOS:~/cablemodem-status# vi
root@CasaOS:~/cablemodem-status# vi /root/cablemodem-status/retriever.py
root@CasaOS:~/cablemodem-status# vi
root@CasaOS:~/cablemodem-status# python3 retriever.py
INFO:app:Reading configuration
INFO:app:Connecting to InfluxDB
INFO:app:Connecting to Selenium remote
INFO:app:Logging into modem
INFO:app:Getting modem status
Traceback (most recent call last):
File "/root/cablemodem-status/retriever.py", line 22, in wrapper
return job_func(*args, **kwargs)
File "/root/cablemodem-status/retriever.py", line 44, in collectionJob
modem.collectStatus()
File "/root/cablemodem-status/modems/netgear_cm2000.py", line 163, in collectStatus
upstreamQamPoints = self.formatUpstreamQamPoints(upstreamQamChannels, sampleTime)
File "/root/cablemodem-status/modems/netgear_cm2000.py", line 54, in formatUpstreamQamPoints
.tag("symbolRate", int(row[4]))
ValueError: invalid literal for int() with base 10: '2560 Ksym/sec'

This should be fixed in the the latest release now: 0.0.21.


SC-10-004 WP2: Password hash contained in exported user data (Info)

This issue is imported from pivotal

https://github.com/OpenConext/OpenConext-myconext/commit/72559540fc31165f5be3ea6f93d4df36f7402478 (Okke Harsta - Aug 26, 2022)


handle WeChat username conflicts

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.roobtyan seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

@roobtyan fix:

@roobtyan fix:

solve it


Misaligned methods are either successfully compiled, or reported with unrelated messages

Case 1
Repro steps
Provide the steps required to reproduce the problem
Take this code:

Expected behavior
It should complain at the declaration of test 2.
Actual behavior
It compiles without errors or warnings.

Case 2
Now, this case isn't actually that bad. What had me and a colleague stumped for some time was the behaviour, when you add a loop to the method:

If I remove two spaces each from the method declaration, it compiles.
In our more complex example, there was an error this value is not a function and can not be applied in the middle of the method, but if you copied the whole code block into another method, it compiled. Removing the block also made the method compile.
Version

Code
module Tests

open System.IO
open System.Xml.Linq
open System
open System.Xml
open System.Security.Cryptography
open Xunit

type FSharpTests() =
    [<Fact>]
    member __.``test 1``() =
        ()

      [<Fact>]
      member __.``test 2``() =
        let folder = Path.Combine(__SOURCE_DIRECTORY__, @"..\..\Data")
        let paths = [| Path.Combine(folder,  "1")
                       Path.Combine(folder, "2") |]
        ignore paths.Length
//        for x in paths do ()
//        ()

Closing in favour of language tracking suggestion https://github.com/fsharp/fslang-suggestions/issues/585


Collections - Ad hoc shareable albums

Problem:
I see a number of requests related to albums and sharing, including:

Sharing of individual photos/videos (https://github.com/bpatrik/pigallery2/issues/243, https://github.com/bpatrik/pigallery2/issues/497, https://github.com/bpatrik/pigallery2/issues/129
Sharing of ad-hoc selection of photos (https://github.com/bpatrik/pigallery2/issues/301)
Sharing of photo searches (https://github.com/bpatrik/pigallery2/issues/81)

Solution Summary:
I propose a PG2 managed folder at the top level of the Gallery (Images folder) called "Collections".  PG2 would create sub-folders for media collections within this.  Media would be added to these Collections using file hard-links to the original file in the gallery.
Benefits:

Collections are very fast (i.e., same speed as existing Gallery).
Negligible additional storage space is required for the hard-linked "copies" in the Collection directories.  The file system simply adds a pointer to the same physical storage for each "copy".
No changes are made to the original photos
Viewing a Collection is done through the existing Gallery view
Sharing mechanism is unchanged, but now has ability to share arbitrary collections of media.
Photos can be added to an Collection iteratively

MVP:
Core updates:

Create a "Collections" folder inside the root "Images folder".   This is writable by PG2.

Create/Add Photos to Collections:

Add "Save to Collection"  button on the search screen.  This button opens a dialog which allows the user to select an existing Collection or create a new one.  If required, it then creates a folder with that name in the Collections folder.  It then  and adds hard links in that folder to all files in the search.

If the user enters a name that matches an existing Collection, they are prompted to select that Collection.
If any of the added files already exist in the Collection, skip them.

Delete Photos from Collections:

Done through file system

Delete Collections:

Done through file system

NOTES:

Duplicate thumbnails and converted photos/video will be stored for each "copy" of a photo in a Collection.

Backlog Features:
In (arguably) descending priority:

Select individual photo from the Gallery or a Search and add to a Collection (could be a menu option or button on the full screen view that brings up the "Save to Collection" dialog.
Remove individual photos from a Collection (i.e., a Garbage Can icon on the tiles in the Collection).
Delete Collection:  Add ability to delete Collections using the "Garbage can" icon on each Collection tile.  Deletes folder and all hard links in it.
Multi-select photos from the Gallery or Search result and add to Collection.
Multi-select photos from the album and remove them from the Collection.
Clean-up Collections Background Task:  A regularly scheduled task to remove hard-linked photos in Collections if the original/source file in the Gallery is deleted (otherwise the link(s) in the Collection folder(s) will retain the file).
Logic to create hard-links to thumbnails, converted photos/videos, etc for file instances in Collections, rather than creating duplicates.
Move "Collections" folder outside the Images folder directory structure and create a new "Collections" item in the top task bar.  This would open a separate Gallery-style view at the Collection folder root.

Summary:
This appears to be an approach that at first glance offers a lot of flexibility while exploiting existing capabilities.  Of course, I don't know the code, so there may be obstacles that aren't apparent to me.
Note that you can implement Collections directly through the file system by manually creating the "Collections" directory tree and adding hard-links..  However, it would be much more convenient to have this integrated into BG2.

Thank you for your FR.
If the app should support #301, it should do it properly. Implementing that would take a long time and not part of my current main focus.
I'm currently working on creating extension support with the app. The solution described above, should be a possibility with my extension support that i'm planning.


New_modes bugfix.

Description of the Change:
Replaces the ops.New singleton object with a similarly named function that creates a New_modes instance and appends it to the Program.
Benefits:
Previously, since there was just a single object, the New_modes.n instance variable would be overwritten each time New.__call__ was called. New_modes.n is currently only used by New_modes.__str__, which is why this bug escaped notice.
This should be a drop-in replacement.

@smite can you clarify what you mean by "there was just a single object"?

@smite can you clarify what you mean by "there was just a single object"?

Before ops.New was a pre-constructed instance of the class ops.New_modes. It was callable via its __call__ method. You always used the same object, so when it stored the number of modes created in self.n, the previous value would always be overwritten. Now the ops.New() function constructs a new ops.New_modes instance each time it's called.

@smite got it, make sense

Ready for merging from my perspective. @josh146 any last issues?

Looks good from my end. Weird that code factor doesn't seem to be working.


php7-fileinfo missing

Hello,
Seems like php7-fileinfo ext is missing.
 See PR : https://github.com/mage2click/m2c/pull/30


Merge pull request #1 from codecombat/master

I

Coverage remained the same at 61.91% when pulling c55e6c4f6904b3a425b3941620c5dc93e22b3061 on jay20162016:master into f0f359046a1b607563a1b577a4a96a6c51184c35 on codecombat:master.


CB-8933: CIS-Harden SSH Configurations_part2

#Part2 of SSH hardening.
Below changes has been enforced in this PR
5.2.11 Ensure only approved ciphers are used
Ciphers aes256-ctr,aes192-ctr,aes128-ctr
5.2.12 Ensure only approved MAC algorithms are used
MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,umac-128-etm<EMAIL_ADDRESS>

Looks like this was addressed in separate PR...

Looks like this was addressed in separate PR...


Build LIT Animated Carousel

Build and implement a LIT Carousel
Example: https://lit.dev/tutorials/carousel/

Hey can you assign me this and in which file do I have to make changes.

Assigned! You can create a new directory for the carousel files in components/LIT
Something like components/LIT/Carousel would be good. Thanks and good luck!


Update Publishing to Docker ACOM article

Currently we point to https://docs.microsoft.com/en-us/azure/vs-azure-tools-docker-hosting-web-apps-in-docker, but I think this content is outdated. @spboyer what should we be pointing to?

This is the correct link, but needs to be updated.

There is not a way to create and issue in the azure-docs repo? Need to get the referenced article updated, the powershell script in the tools is no longer there and is invalid.
This is also for VS 2015, should have one for VS 2017 as well.
@squillace

Under review with @danroth27 and @stevelasker

@danroth27 - @mlearned said he would update the article (owner)
cc:/ @erickson-doug

@mlearned said he would update the article (owner)

cc:/ @erickson-doug
@mlearned  can we get an ETA on this update? Can you mention me in the PR? Can you create an issue I can point to?
cc @spboyer

just replied via email.  I'm full until week of Dec 4th if I need to update it.  (focused on Azure DevOps project and other commitments) ..I could carve some time that week to roll something out if needed.

@mlearned said he would update the article (owner)

cc:/ @erickson-doug
@mlearned  can we get an ETA on this update? Can you mention me in the PR? Can you create an issue I can point to?
cc @spboyer

@mlearned said he would update the article (owner)

cc:/ @erickson-doug
@mlearned  can we get an ETA on this update? Can you mention me in the PR? Can you create an issue I can point to?

@Rick-Anderson  sorry I had a couple articles confused and this one slipped through cracks. I'm not sure on original owner or how it got to me, but I'm updating via this work item here to knock it out https://mseng.visualstudio.com/TechnicalContent/_workitems/edit/1303910

@Rick-Anderson  can you elaborate on the updates needed for this one?  it was updated back in may for VS 2017 … are there more updates needed?  https://docs.microsoft.com/en-us/azure/vs-azure-tools-docker-hosting-web-apps-in-docker       … sorry for back/forth, there are a lot of articles in play and want to make sure we are on the same page here.

@mlearned  closing now, I didn't know it was updated in May.


codebeat badge

Is it fine to add codebeat badge to README?
codebeat is automated code review tool for Swift,Ruby,Go & Python that helps get instant feedback on code quality.
"Quick wins" suggested by codebeat could be a nice candidate for a pull request and help other developers become contributors.
FYI. To be fully open and honest. I'm co-founder of codebeat.

Hi @korzonek! Sure thing, I'm curious to see what it has to suggest.

@jszwedko Thank you!


Orderbook is scrollable, latest trade price at the center

It's similar to the order book on Kucoin exchange

We need add price in dollar to latest price


[ENH] Add a summary of the important imaging parameters found in input data

I think it could be useful to have a TSV in the output that summarizes important parameters in the input datasets and describes how they are included in the results. Potentially something like:

BIDSFile
PhaseEncodingDirection
PartialFourier
TotalReadoutTime
SamplingSchemeType
FinalFile

sub-X_ses-Y_dwi.nii.gz
[ijk-]
[float]
[float]
{"shelled", "cartesian", "random"}
sub-X_ses-Y_desc-preproc_dwi.nii.gz

Some background: we have a new Gibbs unringing method in the next release that specifically works with partial fourier acquisitions. If a user selects --combine-before-denoising we need to determine what partial fourier value to set for the combined scan (maybe exit with an error). So a table like this will need to be made during the pipeline building anyways.
Also, since it would be written to the outputs before the pipeline is run it could provide some hints about possible BIDS issues.
Any thoughts on this idea @cookpa @smeisler @tsalo @arokem

I think this is a good idea, but it might also be good if these were included in the final *preproc_dwi.json files too if they are or already, since that way these fields could be used in BIDS loader filters. I’m not sure I see the immediate need to filter based on these, but still good to have the option. Not sure if that would require an extension of the DWI BIDS specification.

I'm thinking this file will go into the DWI Derivatives - is there an official spec for that?


validation items

[ ] load participation factors sum to 1 in a region/zone (CDM TableParsing Parsing)
[ ] load time series has values 0-1.0
[x] see #447

Add to this list:

[ ] check active and reactive power values are consistent according to the status field and also the ranges. Requires merging #288
[ ] Failing CI in windows due to issues with the name.

@claytonpbarrows what's the status of this? Can we close the issue or do we have pending items?

These have not been addressed. There remain a number of validation checks that we could add to this list (e.g. network connectivity)

@claytonpbarrows I will close this issue since it is stale and hasn't seen change in ver a year.


multi-module Maven Plugin Performance

The plugin re-initializes itself for each child project - this includes checks for updates, etc.
In addition to fixing some of the issues with re-initialization consider limiting the number of times per day the NVD is checked....

After continued testing I have decided to close this issue as I believe the goal has been achieved.


HADOOP-16128 Fix FileSystem.newInstance resource leaks in unit tests

Motivation -
Fix resource leak created in unit tests that instantiate FileSystem resource.
Result -
Fixed the resource leaks in the unit test for the S3A module. Run all unit tests locally successfully.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
12m 23s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 2 new or modified test files.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
30m 50s

trunk passed

+1 :green_heart:
compile
0m 47s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
0m 40s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
0m 32s

trunk passed

+1 :green_heart:
mvnsite
0m 47s

trunk passed

+1 :green_heart:
javadoc
0m 27s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
0m 35s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
1m 12s

trunk passed

+1 :green_heart:
shadedclient
14m 21s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 35s

the patch passed

+1 :green_heart:
compile
0m 36s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
0m 36s

the patch passed

+1 :green_heart:
compile
0m 45s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
0m 45s

the patch passed

-1 :x:
blanks
0m  0s
/blanks-eol.txt
The patch has 7 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

-0 :warning:
checkstyle
0m 26s
/results-checkstyle-hadoop-tools_hadoop-aws.txt
hadoop-tools/hadoop-aws: The patch generated 24 new + 2 unchanged - 0 fixed = 26 total (was 2)

+1 :green_heart:
mvnsite
0m 37s

the patch passed

+1 :green_heart:
javadoc
0m 16s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
0m 26s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
1m 12s

the patch passed

+1 :green_heart:
shadedclient
14m  2s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 13s

hadoop-aws in the patch passed.

+1 :green_heart:
asflicense
0m 35s

The patch does not generate ASF License warnings.

85m  4s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3237/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3237

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux fc7c4833414b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 11c287b6e03e712fb582e6dc191de81bfdef19ec

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3237/1/testReport/

Max. process+thread count
543 (vs. ulimit of 5500)

modules
C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3237/1/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

I am fixing the build failures. Will push another revision.


Page to "Apply as a Mentor"

In the website, there is a button "Apply Here" as a Mentor, but when I click on it, its staying in the same page instead of getting redirected to a new page. I would like to work on that page.

hey @anishacse2018 nice to see your attention for the issues. We need a new form for that. Go ahead and create a page for it that contains a form ith name email skills and a drive link section to upload CV

Yeah, I will start working on it. Thanks.

I have completed this task. Could you review the code and tell me the next steps.


Upgrade y18n dependency

This PR:

Upgrades y18n from 4.0.0 to 4.0.1 to fix prototype pollution vulnerability in msal-browser and msal-react

Coverage increased (+12.2%) to 96.024% when pulling 9091e950cd91168fac6a173d039c1091213e632f on update-y18n into 0bf4bfc4b8c7ac6ea11b43eceaf406308418c231 on dev.


Please add scheduling support

Please add scheduling support or some argument that automatically starts the download process.
I'd also like to not take up another ticket so I'd also like to see download validation support (where duplicate downloads are rejected without being downloaded) as soon as you are able.
Thanks.

Hi @AnonymousUser200102010
Yeah basically your OS should schedule when youtube-dl-gui should run. I don't really want to turn youtube-dl-gui into a daemon. I am planning to add support for command line arguments where you will be able to specify a file as the input with the URLs you want to download. Then you will be able to create a batch file and run it using the OS scheduler.
Now for the second issue, youtube-dl-gui already checks if the video is presented before it starts the download process. Also, you should check the --download-archive FILE option see: https://github.com/rg3/youtube-dl/blob/master/README.md#options for more informations i think thats exactly what you ask for.

Closing this feel free to reopen if you have to


Removed async, cheerio, url-regex, xss-filters, gulp-dest in package.json

Pre-Submission Checklist

[x] Your pull request targets the staging branch of FreeCodeCamp.
[x] Branch starts with either fix/, feature/, or translate/ (e.g. fix/signin-issue)
[x] You have only one commit (if not, squash them into one commit).
[x] All new and existing tests pass the command npm run test-challenges. Use git commit --amend to amend any fixes.

Type of Change

[x] Small bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds new functionality)
[ ] Breaking change (fix or feature that would change existing functionality)
[ ] Add new translation (feature adding new translations)

Checklist:

[x] Tested changes locally.
[x] Closes currently open issue: Closes #10395

Description
Removed the obsolete dependencies:
async,
cheerio,
url-regex,
xss-filters,
gulp-dest
from the file package.json

:sparkles: :sparkles: LGTM :+1: :sparkles: :sparkles:
:tada: Thanks for contributing @stancheta and congratulations on your first PR submission to FreeCodeCamp! :tada:

Thanks @atjonathan!


Colab example does not lead to reduced memory footprint, leading to OOM errors when loading T5-11B on T4 GPU

Hello,
Thanks a lot for this great library and all your effort regarding it, it is much appreciated :-)
I am trying to replicate the basic example from https://colab.research.google.com/drive/1YORPWx4okIHXnjW7MSAidXN29mPVNT7F
Error / Problem
Loading t5-11b crashes with OOM Error and loading t5-3b works, but get_memory_footprint() returns<PHONE_NUMBER> instead of the expected<PHONE_NUMBER> . This is 1.8 times larger, leading me to believe 8bit is not used at all. However, the stacktrace at crash when loading the 11B model shows it is used after all:

Loading checkpoint shards:  67%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████                                                                  | 10/15 [00:28<00:14,  2.83s/it]
Traceback (most recent call last):
File "transformer_test.py", line 3, in
model_8bit = AutoModelForSeq2SeqLM.from_pretrained("ybelkada/t5-11b-sharded", device_map="auto", load_in_8bit=True)
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 463, in from_pretrained
return model_class.from_pretrained(
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/transformers/modeling_utils.py", line 2476, in from_pretrained
) = cls._load_pretrained_model(
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/transformers/modeling_utils.py", line 2788, in _load_pretrained_model
new_error_msgs, offload_index, state_dict_index = _load_state_dict_into_meta_model(
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/transformers/modeling_utils.py", line 665, in _load_state_dict_into_meta_model
set_module_8bit_tensor_to_device(model, param_name, param_device, value=param)
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/transformers/utils/bitsandbytes.py", line 71, in set_module_8bit_tensor_to_device
new_value = bnb.nn.Int8Params(new_value, requires_grad=False, has_fp16_weights=has_fp16_weights).to(device)
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/bitsandbytes/nn/modules.py", line 202, in to
return self.cuda(device)
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/bitsandbytes/nn/modules.py", line 166, in cuda
CB, CBt, SCB, SCBt, coo_tensorB = bnb.functional.double_quant(B)
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/bitsandbytes/functional.py", line 1529, in double_quant
out_row = torch.zeros(A.shape, device=device, dtype=torch.int8)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 16.00 MiB (GPU 0; 14.56 GiB total capacity; 14.05 GiB already allocated; 16.44 MiB free; 14.05 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

The code

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
import torch
model_8bit = AutoModelForSeq2SeqLM.from_pretrained("ybelkada/t5-3b-sharded", device_map="auto", load_in_8bit=True)
print(model_8bit.get_memory_footprint())

My setup
I am testing on a freshly setup Ubuntu 18.04 LTS with a T4 GPU, the 510 nvidia drivers, CUDA 11.6 and PyTorch 1.13.1. cuDNN is v8.6 as v8.7 did not have a Ubuntu 18.04 installer. Python version is 3.8

Thanks for the report. It seems that there is some sort of incompatibility between the main branch of transformers and the latest version of accelerate that I need to investigate more.
To temporary fix that you can either install the latest release of transformers & accelerate (i.e. not the lib from source) or install both of them from source
pip install git+https://github.com/huggingface/transformers.git
pip install git+https://github.com/huggingface/accelerate.git

I have also updated the colab accordingly: https://colab.research.google.com/drive/1YORPWx4okIHXnjW7MSAidXN29mPVNT7F?usp=sharing
Let me know if this works!

Actually this is related to a recent PR that has been merged for T5 models
For very large T5 models, it appeared that some models led to inconsistent results / output when using them in half precision (float16), therefore - only for T5 - we had to manually cast some layers in full precision (fp32) to ensure stability at inference. This also affects int8 models since now for T5, by default all wo layers are upcasted to float32 (therefore not converted in 8bit)
Since the inference was already stable for t5-3b and t5-11b, you can disable this behaviour for the models provided on the examples and get the expected results with the following modifications:
from transformers import T5ForConditionalGeneration, AutoTokenizer
import torch

# T5-3b and T5-11B are supported!
# We need sharded weights otherwise we get CPU OOM errors
model_id=f"ybelkada/{model_name}"

tokenizer = AutoTokenizer.from_pretrained(model_id)
T5ForConditionalGeneration._keep_in_fp32_modules = None
model_8bit = T5ForConditionalGeneration.from_pretrained(model_id, device_map="auto", load_in_8bit=True)

Related issue: https://github.com/huggingface/transformers/issues/20287

@younesbelkada
Thank you so much for your in-detail response :-)
Actually the issue you referenced was the reason I was installing transformers from source, as the model I am trying to use is Flan-T5-XXL.
Did I understand correctly that it should work when installing transformers and accelerate from source?
And that, for the "vanilla" T5 models, installation from source is not needed, but _keep_in_fp32_modules should be set?
I will try to and report back later today or tomorrow

Thank you for your reply!

So for using flan-t5-xxl, installing transformers & accelerate should just work fine indeed (but I suspect the GPU RAM on Colab might be not enough)
for the vanilla T5 models you can either:
1- install transformers & accelerate from source but add T5ForConditionalGeneration._keep_in_fp32_modules = None if you want to have the same memory footprint as the one stated on the Colab. Otherwise it'll run with the memory footprint that you have shared (around 4-5GB instead of 3GB)
2- or just install these libraries using pip and no additional code should be added (at this time of writing)

Note again that if you want to use vanilla T5 with minimal GPU memory (i.e. with the same memory footprint as the one on the colab), upcasting the layers in full precision seems to be not needed at all.
The issue seems to appear only for large T5 models, more particularly flan-t5-xxl that we advice users to upcast the wo layers in full precision for maximal precision.

@younesbelkada
After installing the libraries from source (getting accelerate==0.15.0.dev0, transformers==4.26.0.dev0) the t5-11b model indeed loads fine on a T4 as long as T5ForConditionalGeneration._keep_in_fp32_modules = None is set. Thank you for that!
For the FLAN version, however, the following code crashes:
from transformers import T5ForConditionalGeneration, AutoTokenizer
import torch

T5ForConditionalGeneration._keep_in_fp32_modules = None # WITHOUT THIS A T4 GPU WILL RUN INTO OOM

model = T5ForConditionalGeneration.from_pretrained("chenz16/flan-xxl-sharded-fp16", device_map="auto", load_in_8bit=True)
tokenizer = AutoTokenizer.from_pretrained("google/flan-t5-xxl")

input_ids = tokenizer("translate English to German: Hello my name is Younes and I am a Machine Learning Engineer at Hugging Face", return_tensors="pt").to("cuda").input_ids
outputs = model.generate(input_ids, max_new_tokens=128)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

with

Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 12/12 [00:18<00:00,  1.52s/it]
/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/bitsandbytes/autograd/_functions.py:231: UserWarning: MatMul8bitLt: inputs will be cast from torch.int8 to float16 during quantization
warnings.warn(f"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization")
Traceback (most recent call last):
File "fintest.py", line 10, in
outputs = model.generate(input_ids, max_new_tokens=128)
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/transformers/generation/utils.py", line 1199, in generate
model_kwargs = self._prepare_encoder_decoder_kwargs_for_generation(
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/transformers/generation/utils.py", line 608, in _prepare_encoder_decoder_kwargs_for_generation
model_kwargs["encoder_outputs"]: ModelOutput = encoder(**encoder_kwargs)
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/accelerate/hooks.py", line 156, in new_forward
output = old_forward(*args, **kwargs)
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/transformers/models/t5/modeling_t5.py", line 1053, in forward
layer_outputs = layer_module(
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/accelerate/hooks.py", line 156, in new_forward
output = old_forward(*args, **kwargs)
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward
hidden_states = self.layer-1
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/accelerate/hooks.py", line 156, in new_forward
output = old_forward(*args, **kwargs)
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/transformers/models/t5/modeling_t5.py", line 334, in forward
forwarded_states = self.DenseReluDense(forwarded_states)
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/accelerate/hooks.py", line 156, in new_forward
output = old_forward(*args, **kwargs)
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/transformers/models/t5/modeling_t5.py", line 317, in forward
hidden_states = self.wo(hidden_states)
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/accelerate/hooks.py", line 156, in new_forward
output = old_forward(*args, **kwargs)
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/bitsandbytes/nn/modules.py", line 260, in forward
out = bnb.matmul(x, self.weight, bias=self.bias, state=self.state)
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/bitsandbytes/autograd/_functions.py", line 403, in matmul
return MatMul8bitLt.apply(A, B, out, bias, state)
File "/root/miniconda3/envs/HugEnv/lib/python3.8/site-packages/bitsandbytes/autograd/_functions.py", line 321, in forward
output += torch.matmul(subA, state.subB)
RuntimeError: "addmm_cuda" not implemented for 'Char'

Any further help is much appreciated, thank you in advance.

@younesbelkada Any update on this? :-)

Ran into the same error when running the same code. If I'm interpreting this right, before passing input to a wo layer (or at least one type of them, specifically T5DenseGatedActDense), the transformers library moves it to the same dtype as the wo layer (see here) which is expected to be in torch.float32 to avoid the whole non-sensical output issue. But if you keep wo layers in 8bit to avoid OOM errors, then the input is moved to torch.int8, which wo layers apparently just cannot handle ("addmm_cuda" not implemented for 'Char'). Keeping the wo layers in 8bit and forcing its input to be torch.float16 can avoid runtime errors but this approach still leads to non-sensical output that upcasting to torch.float32 was supposed to solve in the first place.
I'm assuming this means that T5ForConditionalGeneration._keep_in_fp32_modules = None is really a no-go for Flan T5 XXL. Any workarounds for this? I assume device_map="auto" isn't supposed to come into play when loading Flan T5 XXL with some modules kept in fp32? Though that's probably a question for accelerate, not bitsandbytes.

Hi @ryan-caesar-ramos @f-lng
your intrepretation is right and this is a bug. It should be addressed in https://github.com/huggingface/transformers/pull/21281 and you'll be able to run the script without the issue described above
However as stated in https://github.com/TimDettmers/bitsandbytes/issues/111#issuecomment-1368873766 , you will most likely get inconsistent output if you disable _keep_in_fp32_modules

@younesbelkada Thanks. According to the mentioned PR, which has been merged into main yesterday, that means that now Flan-T5 should work in 8bit when installing from source? Thats how I understood it, would be great for a confirmation, as I am currently not able to access my test machine.

Hi @f-lng
Yes, but if you use flan-xxl, you will most likely get gibberish output as stated in my previous comments.

@younesbelkada So I understood it wrong, I thought https://github.com/huggingface/transformers/pull/21281 would also address this issue.
Is there any ongoing issue I could follow regarding this or is it not expected to be fixed anytime soon?
So right now for FLAN-XXL the only way to properly run it is using half precision not 8bit, correct?

I figured out that the neccessary T5ForConditionalGeneration._keep_in_fp32_modules = None breaks the model, and on top of that having both the encoder and the decoder fully in 8bit also makes the runtime very slow.
So the only method of running T5-11B while still maintaining its prediction quality and performance is to get another T4, I think there is no way around this if you want to do proper sequence generation (perhaps with small inputs and as an encoder-only model its different).
Anyway, for reference if someone stumbles upon this issue in the future, this is what I ended up with in the end: https://github.com/TimDettmers/bitsandbytes/issues/142#issuecomment-1431258698


Typo in 'Loading in chunks' section

There is a small typo in the Loading in chunks section of the README:
import { ChangeEvent } from '@angular2-virtual-scroll';

should be:
import { ChangeEvent } from 'angular2-virtual-scroll';

(without the @)

Fixed in github. I won't publish to npm until the next version (unfortunately even for documentation changes you have to update the version #). Thanks.


Increase timeout to handle ident

The bridge can take up to 30 seconds to do ident. The
connect timeout has been raised to 45 seconds.
Fixes #2

Holy s*** you're fast!

It's SOCKET_TIMEOUT from my testing for waiting for the prompt:
E.g.: https://github.com/bdraco/aiolip/blob/master/aiolip/protocol.py#L46


[WIP] docs: Consolidate builders

Commit Message: docs: Consolidate builders
Additional Description:
Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Deprecated:]
[Optional API Considerations:]

docs are rendered here https://storage.googleapis.com/envoy-pr/18178/docs/index.html


{azure bastion} Remove dependency on msazurerest

This checklist is used to make sure that common guidelines for a pull request are followed.
Related command

General Guidelines

[ ] Have you run azdev style <YOUR_EXT> locally? (pip install azdev required)
[ ] Have you run python scripts/ci/test_index.py -q locally? (pip install wheel==0.30.0 required)
[ ] My extension version conforms to the Extension version schema

For new extensions:

[ ] My extension description/summary conforms to the Extension Summary Guidelines.

About Extension Publish
There is a pipeline to automatically build, upload and publish extension wheels.
Once your pull request is merged into main branch, a new pull request will be created to update src/index.json automatically.
You only need to update the version information in file setup.py and historical information in file HISTORY.rst in your PR but do not modify src/index.json.

Thank you for your contribution! We will review the pull request and get back to you soon.

/azp run

@aavalang Pls sync with latest cli dev branch and re-record failed tests. There's API version mismatch issue

@evelyn-ys @yonzhan can you please merge this?


Restructure project directory layout and exception hierarchy

This PR restructures the project directory layout and exception hierarchy to bring them in line with common practices and provide a better foundation for custom exceptions and logging. It follows up on discussions in pydot/pydot#171 and pydot/pydot#230.
See the individual commit messages and new ChangeLog entries for additional remarks. I won't copy them here.
Reviews/comments welcome. I won't merge it before 2021-07-26 anyway, later if necessary to resolve any comments.

@ankostis: In response to some of your comments of last year:

The Exceptions should be definitely defined in the package-module.

I now define them in exceptions.py and in __init__.py do from pydot.exceptions import *. That comes down to the same as what you meant, right?

It is tempting to copy-paste the whole pydot.py contents on the package-module
but i would recommend against it.

Hmm, I am not sure if I followed up this advice completely. Taken literally, at least I did not copy-paste everything into __init__.py. Only __author__, __version__ and __license__. However, I did rename pydot.py to core.py and then added from pydot.core import * to __init__.py, which basically comes down to importing "the whole pydot.py" after all. Not sure if you meant to advise against that as well, but I do not really see how we can avoid that without breaking things. See example below.

TIP1:
When a user imports a module further into a package, the package-module gets import
in its entirety - so if it is "heavy" it delays the import.  That is important when
the different functionality is split in multiple submodules of the package.
Example:
A package contains 3 sub-modules: base.py, sphinx_extension.py
If base.py contains also code for irrelevant http-server communications,
whenever from <package> import sphinx_extension code is trun,
it has to pay the price for this never-to-be-used http-server functionality.

TIP2
The modules are split based on the 3rd-party imports.  If a module is completely disparate,
it might become an "extras" in the setup.py.

I have been thinking about how to apply this to pydot, but run into objections regarding backwards compatibility and user-friendliness.

The obvious candidate to be split off from core is the parser-related code, which includes a dependency on PyParsing. Most of it is already in a separate module (dot_parser.py). There are only two functions in core.py that depend on dot_parser: graph_from_dot_data() and graph_from_dot_file(). If we would move those out of core.py and drop the import of dot_parser, the loading time for pydot would be cut by 46% (from 89 ms to 48 ms, as measured by python -X importtime -c "import pydot", Python 3.7.3, Celeron N3350, lowest of multiple runs). This reduction is mostly (98%) attributable to not importing pyparsing anymore.
However, I am not very eager to remove parsing from the standard functionality offered by pydot. I mean, currently import pydot means you get access to all its functionality immediately. If we take the parsing functionality out, users would need to start thinking about how to import which part of pydot. To use the parsing functions they would need to do import pydot.dot_parser, from pydot import dot_parser or from pydot.dot_parser import graph_from_dot_data, graph_from_dot_file. This is not only a breaking change for existing scripts (we could at best smooth the transition by adding a deprecation warning in pydot 2.0 and postponing the removal from core to pydot 3.0), but it is also inconvenient for new users and new scripts. An additional import would be needed, different prefixes need to be used (e.g. pydot.dot_parser.graph_from_dot_data()) and in the interactive interpreter, using tab completion on pydot. would not show these functions anymore for easy discovery of what pydot offers.
So, I was thinking if instead we could maybe just turn it around: Keep the parsing as default functionality of pydot, but offer users that are looking to reduce load time a way to manually import only the (core) modules they need. However, I am getting the impression that this is simply impossible, because __init__.py always gets executed, even if the user only imports a specific module inside the package. We need __init__.py to carry the parsing stuff if we want to offer that as default functionality of pydot, but then we do not want that to load when a user chooses to import only the core and exceptions modules for example. Those two wishes are simply incompatible with each other, right? Hope you understand what I am trying to say and please correct me if I am wrong.
Alternatively, if making pydot lighter is the goal, we could just "lazy import" dot_parser by moving the import to inside the function graph_from_dot_data(). A lot can be said about this practice, see this SO question for example. I think we can get over the style objections, and as I understand the performance overhead of repeated import statements for an already imported module is minimal. However, a more important issue may be that any problems met during the import will only show up once the function is called. On the other hand, I assume parsing is one of the first steps in most user scripts, so any import problems should still show up early on. Hope to hear your thoughts. This can also be done separate of this PR.

One might also consider splitting off the functions related to calling Graphviz to a separate module. However, when it comes to offering users the option not to load that module, similar questions as for splitting off the parsing will come up. Calling Graphviz is the reason most people use pydot and the core pydot class Dot has methods Dot.create() and Dot.write() that depend heavily on those functions. Again, we could move the import to inside Dot.create(), but in this case that would really make import problems show up much too late, because users will likely call Graphviz only at the end of their scripts. Finally, the performance benefit is not as strong here: Not loading Graphviz output functions and not importing os and subprocess reduces pydot load time by around 6%.

Again, hope to hear your thoughts. In both cases, we can also decide about it later. No need to do everything in this PR already.

@ankostis I will merge this now already, because I want to keep things moving, but feel free to still comment on it later. Thank you for all your help and hope you are doing well.


Screenshot event

Hello !
Would you be able to add an event during the screenshot? It'd be great !

I am not sure where you want to inject to, so if you would, please submit a pull request!


feat: add possibiblity to use AWS IAM roles for service accounts

what

To allow usage of AWS IRSA the assume role policy of the created IAM role needs to be adapted, therefore an additional (and optional) statement  for the sts:AssumeRoleWithWebIdentity action was added
To decouple sts:AssumeRole for the Service and the AWS principal types all statements have been split into separate blocks

why

To allow usage of AWS IAM roles inside of EKS AWS
more secure than handling AWS access keys and secrets

references

IRSA
1
2

Is there an update on this @goruha, can we get this merged?

looks promising we are waiting to use this feature in our labs as well , +1 for any merging updates ?


Remove pydantic from Python client.

Because it is making client dependencies hard to work with. And it is not critical on the client side anyway.

Done as of datajunction==0.0.1a68


ubuntu18源码编译失败

分支：master
系统：ubuntu18.04
在执行cmake的时候，出现：
-- Version: 0.0.0
-- Performing Test HAVE_STD_REGEX
-- Performing Test HAVE_STD_REGEX -- success
-- Performing Test HAVE_GNU_POSIX_REGEX
-- Performing Test HAVE_GNU_POSIX_REGEX -- failed to compile
-- Performing Test HAVE_POSIX_REGEX
-- Performing Test HAVE_POSIX_REGEX -- success
-- Performing Test HAVE_STEADY_CLOCK
-- Performing Test HAVE_STEADY_CLOCK -- success
-- tensorflow_framework library found: in TF_FRAMEWORK-NOTFOUND
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
TF_FRAMEWORK
    linked by target "tf_euler" in directory /tmp/Euler/tf_euler

-- Configuring incomplete, errors occurred!
See also "/tmp/build/CMakeFiles/CMakeOutput.log".
See also "/tmp/build/CMakeFiles/CMakeError.log".

执行：
python -c "import tensorflow as tf; print(tf.sysconfig.get_lib())"
结果是：
/usr/local/lib/python3.6/dist-packages/tensorflow
请问，可能是什么原因？

知道问题了,确少了libtensorflow_framework.so包
cd /usr/local/lib/python3.6/dist-packages/tensorflow
ln -s libtensorflow_framework.so.1 libtensorflow_framework.so


Support http(s) URLs to point to collections

-c parameter now supports URLs starting with http
moved code into main method to be able to use await for downloading collections
tested with collection shared with Collection Access Key URL

Thanks @jonico 👍


ARROW-16173: [C++] Add benchmarks for temporal functions/kernels follow-up

Adds benchmarks for ISOCalendar and YearMonthDay kernels.
This is to follow-up #12997 and resolve ARROW-16173.

@ursabot please benchmark command=cpp-micro --suite-filter=scalar-temporal

Can you describe the change in the PR text?

It seems like the problem is present here as well. Please don't merge.

Closing because it has been untouched for a while, in case it's still relevant feel free to reopen and move it forward 👍


[Question] Run CTransformer with oracle linux server hits error with libctransformers.so

issue
Hi, folks, I'm trying to run a script that does some Q&A with a pretrained model loaded using CTransformers.
I'm able to run the code successfully on mac but when it comes to deploying it on a linux server (e.g oracle linux) it seems to break.
The stacktrace i get is:
python /home/opc/fine_tuned/fine_tune.py
/home/opc/miniconda3/envs/myenv/lib/python3.8/site-packages/langchain/__init__.py:34: UserWarning: Importing PromptTemplate from langchain root module is no longer supported. Please use langchain.prompts.PromptTemplate instead.
  warnings.warn(
loaded all libs!
load and prepare sample training data!
Fetching 1 files: 100%|███████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 18001.30it/s]
Fetching 1 files: 100%|████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 3675.99it/s]
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
Traceback (most recent call last):
  File "/home/opc/fine_tuned/fine_tune.py", line 46, in <module>
    llm=CTransformers(model="TheBloke/Llama-2-7B-Chat-GGML",
  File "/home/opc/miniconda3/envs/myenv/lib/python3.8/site-packages/langchain/load/serializable.py", line 97, in __init__
    super().__init__(**kwargs)
  File "/home/opc/miniconda3/envs/myenv/lib/python3.8/site-packages/pydantic/v1/main.py", line 339, in __init__
    values, fields_set, validation_error = validate_model(__pydantic_self__.__class__, data)
  File "/home/opc/miniconda3/envs/myenv/lib/python3.8/site-packages/pydantic/v1/main.py", line 1102, in validate_model
    values = validator(cls_, values)
  File "/home/opc/miniconda3/envs/myenv/lib/python3.8/site-packages/langchain/llms/ctransformers.py", line 72, in validate_environment
    values["client"] = AutoModelForCausalLM.from_pretrained(
  File "/home/opc/miniconda3/envs/myenv/lib/python3.8/site-packages/ctransformers/hub.py", line 175, in from_pretrained
    llm = LLM(
  File "/home/opc/miniconda3/envs/myenv/lib/python3.8/site-packages/ctransformers/llm.py", line 246, in __init__
    self._lib = load_library(lib, gpu=config.gpu_layers > 0)
  File "/home/opc/miniconda3/envs/myenv/lib/python3.8/site-packages/ctransformers/llm.py", line 126, in load_library
    lib = CDLL(path)
  File "/home/opc/miniconda3/envs/myenv/lib/python3.8/ctypes/__init__.py", line 373, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /home/opc/miniconda3/envs/myenv/lib/python3.8/site-packages/ctransformers/lib/basic/libctransformers.so: cannot open shared object file: No such file or directory

Here are the things i've tried so far:

I made sure the python version used in virtual environment (in this case: conda) is correct:

(myenv) [opc@gpt-server fine_tuned]$ python --version
Python 3.8.18

I made sure the .so file existed and have read permission

(myenv) [opc@gpt-server fine_tuned]$ ls -l /home/opc/miniconda3/envs/myenv/lib/python3.8/site-packages/ctransformers/lib/basic/libctransformers.so
-rw-rw-r--. 1 opc opc 1482416 Nov 19 16:28 /home/opc/miniconda3/envs/myenv/lib/python3.8/site-packages/ctransformers/lib/basic/libctransformers.so

I made sure LD_LIBRARY_PATH used by dynamic linker/loader has the path included and ran a sudo ldconfig just in case

(myenv) [opc@gpt-server fine_tuned]$ echo $LD_LIBRARY_PATH
/home/opc/miniconda3/envs/myenv/lib/python3.8/site-packages/ctransformers/lib/basic:/usr/local/lib

wonder if anyone knows what's causing it or why it's not supported in aarch64 architecture? Thanks for any help in advance!
some system attributes:
$ uname -m
aarch64

#11 ?


ENH: Add Derivative.ul() as an alias to underlier (close #183)

Codecov Report

Merging #195 (0e9796f) into develop (d411b52) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##           develop      #195   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           41        41
  Lines         1199      1205    +6
=========================================
+ Hits          1199      1205    +6

Impacted Files
Coverage Δ

pfhedge/features/functional.py
100.00% <100.00%> (ø)

pfhedge/instruments/derivative/american_binary.py
100.00% <100.00%> (ø)

pfhedge/instruments/derivative/base.py
100.00% <100.00%> (ø)

pfhedge/instruments/derivative/european.py
100.00% <100.00%> (ø)

pfhedge/instruments/derivative/european_binary.py
100.00% <100.00%> (ø)

pfhedge/instruments/derivative/lookback.py
100.00% <100.00%> (ø)

pfhedge/nn/modules/hedger.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d411b52...0e9796f. Read the comment docs.


Halt encountered while attempting to view previously downloaded shape data (in Pharo 10 & 11)

Same error in Pharo 10

Apparently, it's an issue with the shape file; cloning and checking out v5.1.1 of https://github.com/nvkelso/natural-earth-vector seems to have fixed the issue! Perhaps the README should be updated to account for this behavior?


Update DataCite schema for all models retrospectively

Production data update
Datacite versioning details
Periodic updates and map it to versioning

related to #1143

Whenever a DOI metadata is updated, its metadataVersion incremented by 1:

However, due to our need to add related identifiers to the schema as dois, e.g., the dois of a sample within a project to its doi metadata, and due to the dois being generated after publishing the projects, we are creating the doi (metadataVersion = 0), and then instantly updating the metadata to add related identifiers, resulting in metadataVersion = 1 being the first version available to the users.
The approach in creating dois is carried on to their update, so whenever we update the doi metadata in nmrXiv, the metadataVersion actually is incremented by 2 instead of 1

More discussion also needed about the old versions of metadata when replaced with newer ones


Feature: Implement Job Functionality

Is your feature request related to a problem? Please describe.
Currently, we are only able to run a single jobs which is not ideal for production use cases since workflows are sometimes nested
Describe the feature you are requesting, as well as the possible use case(s) for it.
Implement Job functionality so that we can run multiple tasks as one job
Indicate the importance of this feature to you.
Must-have
Anything else?
No response

I am not sure about this. We will use Propeller for deploying simple Wasm modules, not complex jobs. Let's put this one low priority for now.

Will be handled in https://github.com/absmach/propeller/issues/25


Investigate Lack of Discvovered Peers

Describe the bug
When running lodecli beacon (eg: syncing on schlesi) with discv5 enabled, no peers are discovered.
Looking at discv5 logs, it appears as if the discovery is never run.
Expected behavior
Expected to discover peers and see numerous discv5 logs as discv5 packets are exchanged
Steps to Reproduce
Configure lodecli with discv5 bootEnrs
run with DEBUG=discv5*

See https://github.com/ChainSafe/discv5/pull/57
Will cut a v0.1.3 after thats merged, should resolve this problem


Renaming gemfirexd* jars to snappydata-store* jars.

Changes proposed in this pull request

Renaming gemfirexd* jars to snappydata-store* jars.
Adding ReleaseNotes.txt to keep a record of changes
that may need to be published.
Added a section in GitHub PR template to update
ReleaseNotes.txt

Patch testing
./gradlew clean buildAll
Other PRs
https://github.com/SnappyDataInc/snappydata/pull/283

Merging this PR and the related one in snappydata after discussing with Sumedh and Yogesh.


Chapter02 - InputStreamSource is abstract; cannot be instantiated

https://github.com/PacktPublishing/Learning-Spring-Boot-2.0-Second-Edition/blob/b190a5fd021f89dd2fddf81757aafb9c9a54ae56/Chapter02/part1/src/main/java/com/greglturnquist/learningspringboot/HomeController.java#L71

It's InputStreamRrsource and it's not abstract.
https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/core/io/InputStreamResource.html

Sorry, my mistake!


Q: Is it possible to build json tree from already UTF8 encoded values?

Is it possible to build a JSON tree from already UTF8 encoded strings?
eg
utf8 encoded string with Ã¦Ã¸Ã¥

I am not sure what you mean. You can parse any valid JSON text with json::parse(), including UTF-8 encoded strings. You can also create a JSON object with type string by calling the constructor with std::string or const char[].

@clarkk Could you please clarify?

just close the issue.. it was late when I wrote it :)


Wrong classpaths in collector-http-config-reference.xml

Should be <committer class="${committer}.core.impl.FileSystemCommitter">
instead of
<committer class="$committer.impl.FileSystemCommitter">
and change delay to
    <delay default="1000" ignoreRobotsCrawlDelay="true"
           class="${http}.delay.impl.GenericDelayResolver">
          <schedule dayOfWeek="from Monday to Friday"
                    time="from 8:00 to 16:30">10000</schedule>
    </delay>

Your fix did not work probably because the variable needed to be surrounded by braces in this case (it probably did not like being close to a dot).  In any case, I changed it to use a new variable called $fsCommitter instead since variables are fully used for every other config portions in that file. Thanks for reporting documentation errors!

I edited the post with correct code that works. Also notice the endtime of schedule. It also throws exception as it's before starttime.

Thanks, I also fixed this.