Update our docs with instructions on where to run vale sync

person-a
:spiral_calendar_pad:  1 hour ago
I added vale sync to my .bashrc file so that it updates our RedHat style locally when I open my terminal every day. Thank you, everyone who worked on making RedHat packages a reality. (edited)
:bulb:
1
person-b
23 minutes ago
Good idea Person-A :slightly_smiling_face:
Wouldn't you also need to cd to the docs dir? I've just put this in my crontab. I am not sure if it's quite correct though :hmm-yes:
@daily cd ~/openshift-docs && vale sync
(edited)
person-A
:spiral_calendar_pad:  8 minutes ago
I'm not sure. It seems to work for me. I'm guessing it might not be a directory-specific command.
@person-c
@person-d
any insights on this?
person-d
7 minutes ago
I think the answer is "it depends" :slightly_smiling_face:
person-d
6 minutes ago
if you have a .vale.ini in your home directory, I think you can call vale sync from anywhere (edited)
person-d
6 minutes ago
otherwise, you have to call it from the specific folder of your project where the vale configuration is. (edited)
person-a
:spiral_calendar_pad:  5 minutes ago
I'll create an issue to add that info to our docs.
person-c
:spiral_calendar_pad:  3 minutes ago
Consider adding the vale sync step in the regular build script that everybody uses daily without thinking about it, rather than relying on a manual setup step that every writer should not forget to implement.
person-a
:spiral_calendar_pad:  1 minute ago
@person-c
which build script?
person-c
:spiral_calendar_pad:  7 minutes ago
The script that everybody runs daily, that belongs to the repository, and which purpose is to automate such things consistently for everybody. I can't imagine a big project doesn't have one.
:+1:
1
person-a
:spiral_calendar_pad:  6 minutes ago
I see. I wasn't sure if you meant a specific one.
person-a
:spiral_calendar_pad:  4 minutes ago
How might we configure Vale to run vale sync when we use Vale?
person-e
3 minutes ago
you could insert a vale wrapper in ~/.local/bin or something that syncs, then runs Vale. (We also have this "problem", of course :slightly_smiling_face:
person-f
2 minutes ago
maybe
alias gfu='git fetch upstream && vale sync'
in ~./bashrc

Vale sync is covered in the docs here:
https://redhat-documentation.github.io/vale-at-red-hat/docs/main/user-guide/installing-vale-cli/
and is mentioned here
https://redhat-documentation.github.io/vale-at-red-hat/docs/main/user-guide/using-vale-cli/


Error with SCons while Compiling Godot

I've never had such a strange error while working on Linux Before. A few days back I reinstalled windows 7 _86 (32 bit) on my PC. Then I installed Python 2.7 and SCons Latest version on my PC.
I'm using Visual Studio C++ Compiler (Build tools) 2017 here.
I get an error here, whenever I try to make a windows Binary.
C:\godot-gles2\godot-gles2>c:/python27/scons.bat p=windows
scons: Reading SConscript files ...

scons: warning: No version of Visual Studio compiler found - C/C++ compilers mos
t likely not set correctly
File "C:\godot-gles2\godot-gles2\SConstruct", line 3, in <module>

scons: warning: No version of Visual Studio compiler found - C/C++ compilers mos
t likely not set correctly
File "C:\godot-gles2\godot-gles2\SConstruct", line 71, in <module>
Configuring for Windows: target=debug, bits=default
Found MSVC compiler: x86
Compiled program architecture will be a 32 bit executable. (forcing bits=32).
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str':
  File "C:\godot-gles2\godot-gles2\SConstruct", line 327:
    detect.configure(env)
  File "./platform/windows\detect.py", line 343:
    configure_msvc(env, manual_msvc_config)
  File "./platform/windows\detect.py", line 196:
    env.Append(CPPPATH=[os.getenv("WindowsSdkDir") + "/Include"])

C:\godot-gles2\godot-gles2>

It never happened with Linux Before.
I'm a newbie with MSVC, so Can anyone help me why this error is coming with SConstruct File of Godot. I don't understand this error.

 Based on the directory name, it looks like you're attempting to build a third-party branch (I may be mistaken).

I'm trying to compile @karroffel 's fork, but the same is with the current master build. I don't use git, I download the code using zip on github

hello anyone managed to find the problem here?

The WindowsSdkDir environment variable appears to not be set for whatever reason. Either set it manually or reinstall Visual Studio and make sure it gets properly set.


fix($animate) : will not skip the animation of ng-hide remove.

According to current ngAnimate code both adding/removing of animate classes and real change classes are done in single requestAnimationFrame , so when ng-hide is removed from the element it will skip the transition.
This is one way of fixing or other way can be just do only adding of classes at same frame with animate for removing the classes wait for one  requestAnimationFrame after applying of animate classes, or we want to go back to the previous behavior that ngAnimate having to wait for one
requestAnimationFrame before CSS classes were added/removed.
closes #12453

Closing this issue since the root cause was something else that has already been fixed: https://github.com/angular/angular.js/issues/12453#issuecomment-130470653

Otherwise thank you @sreeramu for making the PR.


rootDirs support

Currently, when a transpiled file depends on another typescript file, tstl needs to determine:

Where to emit the corresponding Lua file
How to require the corresponding Lua file

As of now, tstl only considers the rootDir compiler option when attempting resolve a relative path from the file doing the importing, to the imported file. If the imported file is not actually relative to rootDir, the whole absolute path sequence is retained in both emission of the Lua file, and when requiring it.
tstl should instead check every path in rootDirs. If it finds that the dependency is relative to one of the rootDirs it should perform the same "relativizing" it does for rootDir.
This will have the advantage of allowing the use of (particularly developement) typescript libraries to be shared across transpiled projects without deeply nested, host-specific emissions arising in tstl output.

I've experimented with that (for #432), and it looks like rootDirs doesn't affect output structure by design (see https://github.com/microsoft/TypeScript/issues/8245). If there's a need for that, we can:

Investigate if that use case is covered by modules (yarn workspaces?) or project references
Document how API can be used to manipulate output structure
Add a new option


Framerate issues 0.4.6/0.4.7

0.4.5 does not seem to have this issue, but getting lots of FPS drops on the newer version.

I'm seeing this as well. No other mods installed.

Sorry, I will resolve it as soon as possible

Version 0.4.8 may have addressed the issue, at least the FPS is no longer low. If there are still any problems, please continue to reply.

Seems to have fixed it.


K8SPXC-645: improve pitr

 ❨?❩
This PR have following improvements :

Now 2 clusters can write to one bucket without any errors.
When we have a "gap" between binlogs, we will write a log message about it to the user
We always read last gtid set from s3 instead of storing it

Test name
Status

commit: https://github.com/percona/percona-xtradb-cluster-operator/pull/790/commits/c347af8b393e6eb473e1c367ca64933553abe7df
image: perconalab/percona-xtradb-cluster-operator:PR-790-c347af8b

Test name
Status

recreate
passed

big-data
passed

storage
passed

scaling
passed

upgrade-haproxy
passed

haproxy
passed

self-healing
passed

restore-to-encrypted-cluster
passed

init-deploy
passed

scaling-proxysql
passed

upgrade-proxysql
passed

security-context
passed

limits
passed

demand-backup
passed

monitoring-2-0
passed

smart-update
passed

affinity
passed

self-healing-advanced
failed

one-pod
passed

auto-tuning
passed

pitr
passed

proxysql-sidecar-res-limits
failed

demand-backup-encrypted-with-tls
passed

operator-self-healing
passed

users
passed

upgrade-consistency
passed

scheduled-backup
failed

tls-issue-self
passed

tls-issue-cert-manager
failed

commit: https://github.com/percona/percona-xtradb-cluster-operator/pull/790/commits/01f0f1f6b34918cad9a12f035592a9c631544670
image: perconalab/percona-xtradb-cluster-operator:PR-790-01f0f1f6

Test name
Status

recreate
passed

big-data
passed

storage
passed

upgrade-haproxy
passed

scaling
passed

haproxy
passed

self-healing
passed

init-deploy
passed

scaling-proxysql
passed

restore-to-encrypted-cluster
passed

security-context
passed

limits
passed

upgrade-proxysql
passed

self-healing-advanced
failed

monitoring-2-0
passed

demand-backup
passed

affinity
passed

smart-update
passed

one-pod
passed

auto-tuning
passed

proxysql-sidecar-res-limits
passed

operator-self-healing
passed

users
passed

pitr
passed

demand-backup-encrypted-with-tls
passed

upgrade-consistency
passed

tls-issue-self
passed

tls-issue-cert-manager
passed

scheduled-backup
failed

tls-issue-cert-manager-ref
passed

validation-hook
passed

commit: https://github.com/percona/percona-xtradb-cluster-operator/pull/790/commits/7e5a610410128cdb447dde12677857f623d593b4
image: perconalab/percona-xtradb-cluster-operator:PR-790-7e5a6104

Test name
Status

upgrade-haproxy
passed

big-data
passed

storage
passed

self-healing
passed

haproxy
passed

self-healing-advanced
failed

init-deploy
passed

limits
passed

scaling
passed

monitoring-2-0
passed

scaling-proxysql
passed

affinity
passed

security-context
passed

one-pod
passed

auto-tuning
passed

proxysql-sidecar-res-limits
passed

users
passed

tls-issue-self
passed

tls-issue-cert-manager
passed

tls-issue-cert-manager-ref
passed

validation-hook
passed

upgrade-proxysql
passed

smart-update
passed

upgrade-consistency
passed

operator-self-healing
passed

commit: https://github.com/percona/percona-xtradb-cluster-operator/pull/790/commits/7e5a610410128cdb447dde12677857f623d593b4
image: perconalab/percona-xtradb-cluster-operator:PR-790-7e5a6104

Test name
Status

storage
passed

self-healing
passed

self-healing-advanced
failed

big-data
passed

upgrade-haproxy
passed

haproxy
passed

init-deploy
passed

limits
passed

monitoring-2-0
passed

scaling
passed

affinity
passed

scaling-proxysql
passed

one-pod
passed

security-context
passed

auto-tuning
passed

proxysql-sidecar-res-limits
passed

users
passed

tls-issue-self
passed

tls-issue-cert-manager
passed

tls-issue-cert-manager-ref
passed

validation-hook
passed

upgrade-proxysql
passed

smart-update
passed

upgrade-consistency
passed

operator-self-healing
passed

commit: https://github.com/percona/percona-xtradb-cluster-operator/pull/790/commits/7e5a610410128cdb447dde12677857f623d593b4
image: perconalab/percona-xtradb-cluster-operator:PR-790-7e5a6104

Test name
Status

upgrade-haproxy
passed

scaling
passed

scaling-proxysql
passed

security-context
passed

big-data
passed

haproxy
passed

init-deploy
passed

storage
passed

limits
passed

self-healing
passed

monitoring-2-0
passed

self-healing-advanced
failed

affinity
passed

one-pod
passed

auto-tuning
passed

proxysql-sidecar-res-limits
passed

users
passed

tls-issue-self
passed

tls-issue-cert-manager
passed

tls-issue-cert-manager-ref
passed

validation-hook
passed

upgrade-proxysql
passed

smart-update
passed

upgrade-consistency
passed

operator-self-healing
passed

commit: https://github.com/percona/percona-xtradb-cluster-operator/pull/790/commits/7e5a610410128cdb447dde12677857f623d593b4
image: perconalab/percona-xtradb-cluster-operator:PR-790-7e5a6104

Test name
Status

scaling
passed

scaling-proxysql
passed

security-context
passed

upgrade-haproxy
passed

storage
passed

self-healing
passed

self-healing-advanced
failed

haproxy
passed

init-deploy
passed

limits
passed

monitoring-2-0
passed

affinity
passed

one-pod
passed

auto-tuning
passed

proxysql-sidecar-res-limits
passed

users
passed

tls-issue-self
passed

tls-issue-cert-manager
passed

tls-issue-cert-manager-ref
passed

validation-hook
passed

big-data
passed

upgrade-proxysql
passed

smart-update
passed

upgrade-consistency
passed

operator-self-healing
passed

commit: https://github.com/percona/percona-xtradb-cluster-operator/pull/790/commits/7e5a610410128cdb447dde12677857f623d593b4
image: perconalab/percona-xtradb-cluster-operator:PR-790-7e5a6104

Test name
Status

upgrade-haproxy
passed

storage
passed

big-data
passed

recreate
passed

scaling
passed

self-healing
passed

restore-to-encrypted-cluster
passed

scaling-proxysql
passed

self-healing-advanced
failed

demand-backup
passed

security-context
passed

pitr
passed

haproxy
passed

demand-backup-encrypted-with-tls
passed

init-deploy
passed

scheduled-backup
failed

limits
passed

monitoring-2-0
passed

affinity
passed

one-pod
passed

auto-tuning
passed

proxysql-sidecar-res-limits
passed

users
passed

tls-issue-self
passed

tls-issue-cert-manager
passed

tls-issue-cert-manager-ref
passed

validation-hook
passed

upgrade-proxysql
passed

smart-update
passed

upgrade-consistency
passed

operator-self-healing
passed

commit: https://github.com/percona/percona-xtradb-cluster-operator/pull/790/commits/7e5a610410128cdb447dde12677857f623d593b4
image: perconalab/percona-xtradb-cluster-operator:PR-790-7e5a6104


UART read line example

I got the UART example working but I have a very hard time to just read a line. I tried this:

    let config = config::Config::new().baudrate(Hertz(115_200));
    let uart_modem = UartDriver::new(
        peripherals.uart1,
        tx,
        rx,
        Option::<gpio::Gpio0>::None,
        Option::<gpio::Gpio1>::None,
        &config,
    )?;
    let mut pin_modem_power = PinDriver::output(peripherals.pins.gpio6)?;

    log::info!("Starting...");

    let mut cli_buf: Vec<u8> = Vec::new();
    loop {
        log::info!("Sending command");
        uart_modem.write("AT\r".as_bytes())?;

        let mut buf: [u8; 1] = [0; 1];
        match uart_modem.read(&mut buf, NON_BLOCK) {
            Ok(bytes_read) => {
                if bytes_read > 0 {
                    let b = buf[0];
                    cli_buf.push(b);
                    log::info!("{}", b);
                    if b == 0xD {
                        log::info!("{:?}", cli_buf);
                        cli_buf.clear();
                    }
                }
            }
            Err(_) => {}
        }
        FreeRtos::delay_ms(1000);
    }

This code logs the correct values but it repeats the loop after every character. In general this approach seems very verbose. Is there a simple example how to read serial input line by line?
Thank you

Not currently possible, you need to do it yourself.
Or you can implement a PR as to what is suggested here.
Closing in favor of the above bug.


hide pagination in grid when there is only one page

I think it's ok to the show the pagination with only one page.

No problem.  I've overwritten the view in my project


Office documents: add paper size

Expected Behavior
Convert Xlsx file to a PDF file
Current Behavior
If the table length is too long,PDF file Documents will change lines
Possible Solution
Steps to Reproduce (for bugs)
1.Convert Xlsx file to a PDF file
Context
Your Environment
Version used:3
Operating System and version:ubuntu 18.10
Link to your project:

Hi, I found that LibreOffice can convert document to html.
eg: "d:\Program Files\LibreOffice\program\soffice"  --convert-to html template.xlsx
Maybe we can solve the problem in other ways

这个方法很不错，您可以试试。我现在用onlyoffice解决我的需求。

This method is very good. You can try it. I now use onlyoffice to address my needs.

For "simple" documents it seems to work, but I wonder if for more "complex" documents it will break some styling during the process.

Have you checked if this got fixed in LibreOffice v6 available from stretch-backports or sid?

Hello, just wondering if there's an update on this. Have some wide excel files i want to specify page size for.

@PrathamSoni you can open a PR to https://github.com/gotenberg/unoconverter.

Sure, would be happy to help you have pointers on how the new option would be propogated to the command/how that would be bubbled up to this package?


Missing tags when encoding multiple qualities with the encode WOH

Describe the bug
When encoding multiple qualities using the encode WOH the media created is not tagged with the suffix. This is a regression introduced beteween Opencast 9 and Opencast 11.
To Reproduce
Steps to reproduce the behavior:

Go to https://stable.opencast.org/admin-ng/index.html#!/events/events
Login as admin
Start the Process upon Upload and Schedule WF on any media you like, and send the file straight to publishing
Multiple Qualities are created in one encode WO. [1] [2] but none of them are tagged as 480p-quality or similar.

Environment Information

Opencast version with minor release Opencast 11.0.0.SNAPSHOT - 1c5fa68 - built on 2022-04-07, 11.4

Likely caused by this change in ComposerServiceImpl.java between Opencast 10 and 11:
-      List<String> tags = profile.getTags();
-      for (String tag : tags) {
-        if (encodingOutput.getName().endsWith(profile.getSuffix(tag)))
+      for (final String tag : tags) {
+        if (inspectedTrack.getURI().getPath().endsWith(profile.getSuffix(tag))) {
           inspectedTrack.addTag(tag);
+        }
+      }

In the line inspectedTrack.getURI().getPath().endsWith(profile.getSuffix(tag)) Opencast set the following values:

tag=mp3
inspectedTrack.getURI().getPath()=/files/collection/composer/248581_0.mp3
profile.getSuffix(tag)=-audio.mp3

So essentially, the inspectedTrack.getURI().getPath() will never end with profile.getSuffix(tag).


Data augmentation

Data augmentation is useful for self-supervised learning. Would be good to add to the training utility.

Addressed in #16


abnormal behavior auto layout in Xcode 8.2.1 and swift 3  !

after adding XLPagerTabStrip to my project and following the examples:
when the project is running I got behavior of swiping gesture different from tapping of titles
in this screenshot I got space between the names and table view cells and this is when view loaded

if I continue in swiping with gesture I got the same layout (the space stay as is)
if I tapped in the title of some table view title the space is gone as this screenshot

if tried to swipe using gesture again we return to the first screenshot with space
forget about the bold black color underneath the buttonBarView.
any suggestions ?!

when I added this lines of code everything worked perfectly 💯
guard isReload else {
        return [child_1, child_2, child_3, child_4, child_5, child_6, child_7, child_8]
    }

    var childViewControllers = [child_1, child_2, child_3, child_4, child_6, child_7, child_8]

    for (index, _) in childViewControllers.enumerated(){
        let nElements = childViewControllers.count - index
        let n = (Int(arc4random()) % nElements) + index
        if n != index{
            swap(&childViewControllers[index], &childViewControllers[n])
        }
    }
    let nItems = 1 + (arc4random() % 8)
    return Array(childViewControllers.prefix(Int(nItems)))
}

override func reloadPagerTabStripView() {
    isReload = true
    if arc4random() % 2 == 0 {
        pagerBehaviour = .progressive(skipIntermediateViewControllers: arc4random() % 2 == 0, elasticIndicatorLimit: arc4random() % 2 == 0 )
    }
    else {
        pagerBehaviour = .common(skipIntermediateViewControllers: arc4random() % 2 == 0)
    }
    super.reloadPagerTabStripView()
}


Some refactoring.

Coverage increased (+0.03%) to 95.341% when pulling b1db1f2757df2ca5dfbcb5956f30f7988fe67465 on FlorianLautenschlager:master into 5a260d7896839b0c9b35d850910973b16bf5575c on ChronixDB:master.


Mod dont load

hi, so i got the mod for 1.19 and 1.18.2 installed forge and put the mod in the mods folder and started the game but the mod didnt loaded and i dont had it in my mods tab tried to uninstall the game,  still no work  after that i used journeymap to see if it would work and i had journeymap working
i dont have what photos to attach isnt a eror the mod just dont load

I used fabric now it works


/var/run issues

Hi
Running OPNsense 17.1.3-amd64
it seems /var/run is cleared on boot which is a problem when running something like quagga.
Quagga expects /var/run/quagga to exist and be owned by quagga user - It doesn't seem to be retained on reboot thus preventing quagga from starting up again - In older versions of opnsense (16.x) this was not a problem.
Is it possible to create a new folder here on boot as a specific user?

Hi Scott,
Are you using /var RAM disk option (e.g. from Nano image)?
Cheers,
Franco

@fichtner add the creation of this directory to the plugin?

@fichtner honestly not sure - i used the ISO installer so ...

@tsumaru720 this setting is under system advanced but you may be faster if you would enter mount into a shell and look for a /var mount.

@fabianfrz
Well, this is interesting because I do not have the option enabled :)
root@vpn1:/var/run # mount
/dev/gpt/rootfs on / (ufs, local, noatime, soft-updates)
devfs on /dev (devfs, local, multilabel)
tmpfs on /tmp (tmpfs, local)
tmpfs on /var (tmpfs, local)
devfs on /var/dhcpd/dev (devfs, local, multilabel)

Bug maybe?

@tsumaru720 is your config setup from scratch or imported?

Imported (It wasn't a ram-disk their either)

@tsumaru720 what does cat /etc/fstab print?

from pfSense? in that case it might be a different interpretation (configs aren't 100% compatible), just save again and reboot to fix.

@fabianfrz
root@vpn1:/var/run # cat /etc/fstab
# Device                Mountpoint      FStype  Options         Dump    Pass#
/dev/gpt/rootfs /               ufs     rw,noatime      1       1
root@vpn1:/var/run #

@AdSchellevis Nope, was imported from OPNsense 16.7.14 :)

Also ticked /tmp RAM Disk and saved / rebooted - /var is still a ram disk

@tsumaru720 unchecked is correct - I am having /tmp a ram disk but not /var because of other issues and it works on my device as expected. I am also using 17.1.3. So your device has a strange behaviour.

Let's run this...
# grep -c 'use_mfs_.*var' /conf/config.xml

(small correction, no -c)

root@vpn1:/usr/local/etc # grep -c 'use_mfs_.*var' /conf/config.xml
1
root@vpn1:/usr/local/etc # grep 'use_mfs_.*var' /conf/config.xml
    <use_mfs_var_size/>

Oh right, that hasn't been used for a long time.
Can you edit config.xml and just remove this?

After reboot, just reapply quagga package, the directory should be there:
# opnsense-revert quagga

@fabianfrz we should add this to the plugin, but details on why/how follow

For old time's sake... removed in 2015 :) 4b9a265f84e

@fichtner that does appear to have fixed the problem :)

Reboot
re-create folder & set permissions
Start quagga
Reboot
still there :)

I assume it's safe to remove use_mfs_tmp_size too?
root@vpn1:/var/run # grep use_mfs /conf/config.xml
    <use_mfs_tmp_size/>
    <use_mfs_tmp>1</use_mfs_tmp>

I wonder how it has survived in my config for so long? :D
I did originally install 15.7 & have upgraded through to 16.7.14. Tried to do 17.1 but ran into disk space issues so it broke & had to reinstall (The issues are my own fault)

Hi Simon,
Yes, use_mfs_tmp_size can be removed too.
I introduced this problem while splitting "/tmp /var MFS" into two separate options while trying to retain backwards compatibility. To be honest, one of the issues here is that we read XML via grep and that is bound to blow up eventually, but it's so early in the system boot that we can't just fire up the whole PHP... I'll issue a fix one way or another for 17.1.4. Thanks for spotting this. :)
Cheers,
Franco

No problem at all - Keep up the good work with OPNsense! :)


fix: Intercom setting anonymous user a common user id

Description

[!TIP]
Add a TL;DR when the description is longer than 500 words or extremely technical (helps the content, marketing, and DevRel team).
Please also include relevant motivation and context. List any dependencies that are required for this change. Add links to Notion, Figma or any other documents that might be relevant to the PR.

Fixes #Issue Number
or
Fixes Issue URL

[!WARNING]
If no issue exists, please create an issue first, and check with the maintainers if the issue is valid.

Automation
/ok-to-test<EMAIL_ADDRESS>:mag: Cypress test results

[!TIP]
🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉
Workflow run: https://github.com/appsmithorg/appsmith/actions/runs/10643548669
Commit: 339a6de6eb95f66277ebe7c63439d9780785ec6f
Cypress dashboard.
Tags: @tag.All
Spec:
Sat, 31 Aug 2024 08:43:30 UTC

Communication
Should the DevRel and Marketing teams inform users about this change?

[ ] Yes
[x] No

/ci-test-limit

/ci-test-limit-count run_count=10 runId=10643554115

/ci-test-limit runId=10643554115

/ci-test-limit runId=10643554115

/ci-test-limit runId=10643554115

/ci-test-limit runId=10643554115

/ci-test-limit runId=10643554115

/ci-test-limit runId=10643554115

/ci-test-limit runId=10643554115

/ci-test-limit runId=10643554115

/ci-test-limit runId=10643554115

/ci-test-limit runId=10643554115

/ci-test-limit-count run_count=10

Closing, as no longer needed.


Update README

Note: this depends on #185

Current version (SNAPSHOT)
some common use cases
subscribe forum hint
removed incomplete revision history

I wold like the README more like a short entry point with practical advices for new users. Its more importent to be in time, then to have a global history. We could provide a link to a label in the closed issues and pull requests.
What do you think?

Thanks @ganskef! Generally looks good to me. I don't believe 1.1.0-beta1-SNAPSHOT is actually up though is it? I'm hoping to actually put up a new version soon. @jekh if you want me to just give you our sonatype/nexus credentials, I'd be happy to do that too.

Yes, I would love to see a beta-1 release to Central and an update to the versions in the readme (I've noticed many folks using the old versions of LP because they are the only ones available in Central).
Since PR #185 contains a breaking change to filters by adding a parameter to one method, we might want to merge that (if everybody is okay with the change) before releasing beta-1, so users don't have to fix that breakage later.

While this is a breaking change, consider to move the client context
from constructor to clientToProxyRequest, too.
Am 20.07.2015 um 04:01 schrieb Jason Hoetger:

Yes, I would love to see a beta-1 release to Central and an update to
the versions in the readme (I've noticed many folks using the old
versions of LP because they are the only ones available in Central).
Since PR #185 https://github.com/adamfisk/LittleProxy/pull/185
contains a breaking change to filters by adding a parameter to one
method, we might want to merge that (if everybody is okay with the
change) before releasing beta-1, so users don't have to fix that
breakage later.
—
Reply to this email directly or view it on GitHub
https://github.com/adamfisk/LittleProxy/pull/219#issuecomment-122728847.

Since the dependency #185 is been merged, should I go on to merge this too?
What should I do with the SNAPSHOT/release issue?

I see no harm merging the README update. When we release beta-1 to Central, we can update the README again.


augmented assignments with larger indices involve unnecessary change of dtype

Code Sample, a copy-pastable example if possible
In [2]: a = pd.Series(1, index=range(3))

In [3]: b = pd.Series(2, index=range(4))

In [4]: a.dtype
Out[4]: dtype('int64')

In [5]: a += b

In [6]: a.dtype
Out[6]: dtype('float64')

In [7]: a.index == b.index
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-8-2fae764c9047> in <module>()
----> 1 a.index == b.index

/home/nobackup/repo/pandas/pandas/indexes/base.pyc in _evaluate_compare(self, other)
   3026                 if isinstance(other, (np.ndarray, Index, ABCSeries)):
   3027                     if other.ndim > 0 and len(self) != len(other):
-> 3028                         raise ValueError('Lengths must match to compare')
   3029                 func = getattr(self.values, op)
   3030                 result = func(np.asarray(other))

ValueError: Lengths must match to compare

Expected Output
My preference would be
In [6]: a.dtype
Out[6]: dtype('int64')

... but if one prioritises coherence with a = a + b, then the expected behaviour would probably be
In [6]: a.dtype
Out[6]: dtype('float64')

In [7]: a.index == b.index
Out[7]: array([ True,  True,  True,  True], dtype=bool)

(outer join).
In [23]: a = pd.Series(1, index=range(3))

In [24]: a = pd.Series(1, index=range(3))

In [25]: a2, b2 = a.align(b)

In [26]: a2
Out[26]:
0    1.0
1    1.0
2    1.0
3    NaN
dtype: float64

In [27]: b2
Out[27]:
0    2
1    2
2    2
3    2
dtype: int64

So ``a+b`` is exactly right

In [18]: b = pd.Series(2, index=range(4))
In [19]: a = pd.Series(1, index=range(3))
In [20]: a+b
Out[20]:
0    3.0
1    3.0
2    3.0
3    NaN
dtype: float64

Looks like this is just taking the a indices (and not aligning). This probably doesn't have as much test coverage. So I would call this a bug in that ``a`` should yield ``a+b`` here.

In [21]: a+=b
In [22] a`
Out[22]:
0    3.0
1    3.0
2    3.0
dtype: float64

True, your proposed solution is coherent with a = a+b. Just one more argument in favour of my preferred one: coherence with df['a'] = b:
In [2]: df = pd.DataFrame(index=range(3))

In [3]: df['a'] = 1

In [4]: b = pd.Series(5, index=range(4))

In [5]: df['a'] = b

In [6]: df
Out[6]:
   a
0  5
1  5
2  5

... then, according to your solution (trivially extended to DataFrames), we would have:
In [7]: df['a'] += b

In [8]: df
Out[8]:
      a
0  10.0
1  10.0
2  10.0
3  NaN

... while according to my proposal, we would have:
In [7]: df['a'] += b

In [8]: df
Out[8]:
      a
0  10
1  10
2  10

Notice that with neither of these df['a'] += b has the same effect of df['a'] = a + b anyway - which is instead true with the currently implemented behaviour. In other words: whatever decision is taken, there is some inconsistency. I like the option of not changing the dtype just because I think it is more useful.

hmm, there might be some logic in an in-place op to always do left-align (only). I don't think we want to make
df['a'] += b change the index as the b should get aligned to the df.index, NOT the union.
so on second thought I don't think this is a bug, but rather an explict design decision.

I agree with you. So coming back to the dtype, OK for not changing it (when the casting is only a consequence of missing values)? Or talking about implementation: OK for calculating the operation on the left axis, and then putting the values in the left series (rather than calculating the operation on the union and then putting only values corresponding to the left axis in the left series)?

@toobaz I dont really recall this code path. If you can do this w/o jumping thru hoops then ok with changing impl to NOT change dtype. might be straghtforward.


Fail to resume operations after connection error.

Version of Ruby: 2.3.3
Version of Kafka: 2.11.1-1.1.0
Version of ruby-kafka: 0.5.5

Hi. I have a simple script that will basically read from a several topics on a (local) source broker, and copy those messages to a (remote) target broker.
The relevant code looks like this.

        def run
                sleep(@backoff)
                mappings = {}
                @backoff = @backoff * 2 + 1
                src = Kafka.new(@cfg['source']['broker'])
                tgt = Kafka.new(@cfg['target']['broker'])
                consumer = src.consumer(group_id: @cfg['source']['client_id'])
                producer = tgt.producer()

                @cfg['mappings'].each_pair do |from, to|
                        consumer.subscribe(from)
                        mappings[from] = to
                end

                n = 0
                consumer.each_batch do |batch|
                        begin
                                batch.messages.each do |msg|
                                        n += 1
                                        producer.produce(msg.value, key: msg.key, topic: mappings[msg.topic])
                                        if n >= 100
                                                producer.deliver_messages
                                                consumer.mark_message_as_processed(msg)
                                                $logger.info "Send #{n} messages"
                                                n = 0
                                        end
                                end
                                if n > 0
                                        producer.deliver_messages
                                        $logger.info "Send #{n} messages"
                                end
                        rescue Exception => e
                                $logger.debug(e.message)
                                raise e
                        end
                end
        end

Due to connectivity issues, the target broker is sometimes unreachable, and the code raises Could not connect to any of the seed brokers: - kafka://x.x.x.x:9092: Connection timed out. This is normal and expected. However, when I call the function again, the code hangs on consumer.each_batch, and never returns a single message, until I kill and restart the entire process.
Not sure if it's relevant, but there is threading involved. I let the thread die, and restart it.
I'm unsure weather I'm doing something wrong, or if the library tries to re-use some state and therefore fails. Any help on this issue would be appreciated.

Hi @l0x-c0d3z, when the consumers don't return anything, are there any suspicious logs? It would be helpful if you can provide the logs when this situation happens, and even more if the logs is at Debug level. One more thing, how did you handle the exception?

The exception is not caught, so the thread just dies. The main thread detects that and restarts it.
I just checked, but the interesting log part is now out of my screen buffer. I tried simulating the behavior by stopping the target kafka for a few seconds, but it does restart properly. I've added a debug Logger to the Kafka instances, and I'll post the logs as soon as it fails again.
Basically, I had the call stack from the connection failed, followed by the message saying the thread had been restarted, and then nothing until I killed it.

Wait a sec. I have just realize that you are using ruby-kafka 0.5.5 right? From the version 0.6.x stable, ruby-kafka introduces a whole new mechanism of message fetching. What do you think about upgrading to newest stable version and retry again on prod to see whether this issue still exist? In addition, Kafka 0.11.1 and 1.1.0 are supported better (with fancy features) in newer ruby-kafka versions.

I'll give it a try and see if it works better.

Hi @l0x-c0d3z, I haven't heard from you for a while. Does the newer version of ruby-kafka work for you?

It took longer than before, but I did have a worker crash a few days ago. Here's the debug log.
I, [2018-10-02T08:57:05.599527 #18366]  INFO -- : Send 100 messages
D, [2018-10-02T08:57:05.604211 #18366] DEBUG -- : Current leader for my.topic/0 is node <IP_ADDRESS>:9092 (node_id=0)
I, [2018-10-02T08:57:05.604436 #18366]  INFO -- : Sending 100 messages to <IP_ADDRESS>:9092 (node_id=0)
D, [2018-10-02T08:57:05.604546 #18366] DEBUG -- : Sending produce API request 9036 to <IP_ADDRESS>:9092
D, [2018-10-02T08:57:05.630403 #18366] DEBUG -- : Waiting for response 9036 from <IP_ADDRESS>:9092
D, [2018-10-02T08:57:06.026155 #18366] DEBUG -- : Received response 9036 from <IP_ADDRESS>:9092
D, [2018-10-02T08:57:06.026608 #18366] DEBUG -- : Successfully appended 100 messages to my.topic/0 on <IP_ADDRESS>:9092 (node_id=0)
D, [2018-10-02T08:57:06.026722 #18366] DEBUG -- : Marking my.local.topic/0:32992948 as processed
I, [2018-10-02T08:57:06.026762 #18366]  INFO -- : Send 100 messages
D, [2018-10-02T08:57:06.031436 #18366] DEBUG -- : Current leader for my.topic/0 is node <IP_ADDRESS>:9092 (node_id=0)
I, [2018-10-02T08:57:06.031673 #18366]  INFO -- : Sending 100 messages to <IP_ADDRESS>:9092 (node_id=0)
D, [2018-10-02T08:57:06.031780 #18366] DEBUG -- : Sending produce API request 9037 to <IP_ADDRESS>:9092
W, [2018-10-02T08:57:06.057798 #18366]  WARN -- : Reached max fetcher queue size (100), sleeping 1s
D, [2018-10-02T08:57:06.059035 #18366] DEBUG -- : Waiting for response 9037 from <IP_ADDRESS>:9092
D, [2018-10-02T08:57:06.486424 #18366] DEBUG -- : Received response 9037 from <IP_ADDRESS>:9092
D, [2018-10-02T08:57:06.486898 #18366] DEBUG -- : Successfully appended 100 messages to my.topic/0 on <IP_ADDRESS>:9092 (node_id=0)
D, [2018-10-02T08:57:06.487009 #18366] DEBUG -- : Marking my.local.topic/0:32993048 as processed
I, [2018-10-02T08:57:06.487054 #18366]  INFO -- : Send 100 messages
D, [2018-10-02T08:57:06.491648 #18366] DEBUG -- : Current leader for my.topic/0 is node <IP_ADDRESS>:9092 (node_id=0)
I, [2018-10-02T08:57:06.491873 #18366]  INFO -- : Sending 100 messages to <IP_ADDRESS>:9092 (node_id=0)
D, [2018-10-02T08:57:06.491981 #18366] DEBUG -- : Sending produce API request 9038 to <IP_ADDRESS>:9092
D, [2018-10-02T08:57:06.519985 #18366] DEBUG -- : Waiting for response 9038 from <IP_ADDRESS>:9092
W, [2018-10-02T08:57:07.058102 #18366]  WARN -- : Reached max fetcher queue size (100), sleeping 1s
W, [2018-10-02T08:57:08.058358 #18366]  WARN -- : Reached max fetcher queue size (100), sleeping 1s
W, [2018-10-02T08:57:09.058640 #18366]  WARN -- : Reached max fetcher queue size (100), sleeping 1s
W, [2018-10-02T08:57:10.058954 #18366]  WARN -- : Reached max fetcher queue size (100), sleeping 1s
W, [2018-10-02T08:57:11.059205 #18366]  WARN -- : Reached max fetcher queue size (100), sleeping 1s
W, [2018-10-02T08:57:12.059477 #18366]  WARN -- : Reached max fetcher queue size (100), sleeping 1s
W, [2018-10-02T08:57:13.059733 #18366]  WARN -- : Reached max fetcher queue size (100), sleeping 1s
W, [2018-10-02T08:57:14.059964 #18366]  WARN -- : Reached max fetcher queue size (100), sleeping 1s
W, [2018-10-02T08:57:15.060255 #18366]  WARN -- : Reached max fetcher queue size (100), sleeping 1s
W, [2018-10-02T08:57:16.060520 #18366]  WARN -- : Reached max fetcher queue size (100), sleeping 1s
E, [2018-10-02T08:57:16.530021 #18366] ERROR -- : Timed out while waiting for response 9038
D, [2018-10-02T08:57:16.530372 #18366] DEBUG -- : Closing socket to <IP_ADDRESS>:9092
D, [2018-10-02T08:57:16.530733 #18366] DEBUG -- : Closing socket to <IP_ADDRESS>:9092
E, [2018-10-02T08:57:16.530881 #18366] ERROR -- : Could not connect to broker <IP_ADDRESS>:9092 (node_id=0): Connection error Errno::ETIMEDOUT: Connection timed out
W, [2018-10-02T08:57:16.530943 #18366]  WARN -- : Failed to send all messages; attempting retry 1 of 2 after 1s
W, [2018-10-02T08:57:17.060779 #18366]  WARN -- : Reached max fetcher queue size (100), sleeping 1s
I, [2018-10-02T08:57:17.531076 #18366]  INFO -- : Fetching cluster metadata from kafka://<IP_ADDRESS>:9092
D, [2018-10-02T08:57:17.531276 #18366] DEBUG -- : Opening connection to <IP_ADDRESS>:9092 with client id ruby-kafka...
W, [2018-10-02T08:57:18.061049 #18366]  WARN -- : Reached max fetcher queue size (100), sleeping 1s
W, [2018-10-02T08:57:19.061344 #18366]  WARN -- : Reached max fetcher queue size (100), sleeping 1s
W, [2018-10-02T08:57:20.061589 #18366]  WARN -- : Reached max fetcher queue size (100), sleeping 1s
W, [2018-10-02T08:57:21.061870 #18366]  WARN -- : Reached max fetcher queue size (100), sleeping 1s
W, [2018-10-02T08:57:22.062152 #18366]  WARN -- : Reached max fetcher queue size (100), sleeping 1s
W, [2018-10-02T08:57:23.062406 #18366]  WARN -- : Reached max fetcher queue size (100), sleeping 1s
W, [2018-10-02T08:57:24.062696 #18366]  WARN -- : Reached max fetcher queue size (100), sleeping 1s
W, [2018-10-02T08:57:25.062989 #18366]  WARN -- : Reached max fetcher queue size (100), sleeping 1s
W, [2018-10-02T08:57:26.063260 #18366]  WARN -- : Reached max fetcher queue size (100), sleeping 1s
W, [2018-10-02T08:57:27.063542 #18366]  WARN -- : Reached max fetcher queue size (100), sleeping 1s
E, [2018-10-02T08:57:27.536340 #18366] ERROR -- : Timed out while trying to connect to <IP_ADDRESS>:9092: Connection timed out
D, [2018-10-02T08:57:27.536550 #18366] DEBUG -- : Closing socket to <IP_ADDRESS>:9092
E, [2018-10-02T08:57:27.536980 #18366] ERROR -- : Failed to fetch metadata from kafka://<IP_ADDRESS>:9092: Connection timed out
D, [2018-10-02T08:57:27.538135 #18366] DEBUG -- : Could not connect to any of the seed brokers:
- kafka://<IP_ADDRESS>:9092: Connection timed out
E, [2018-10-02T08:57:27.538580 #18366] ERROR -- : Exception raised when processing my.local.topic/0 in offset range 32991083..32993293 -- Kafka::DeliveryFailed: Could not connect to any of the seed brokers:
- kafka://<IP_ADDRESS>:9092: Connection timed out
/opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/producer.rb:378:in `rescue in block in deliver_messages_with_retries'
/opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/producer.rb:375:in `block in deliver_messages_with_retries'
/opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/producer.rb:370:in `loop'
/opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/producer.rb:370:in `deliver_messages_with_retries'
/opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/producer.rb:247:in `block in deliver_messages'
/opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/instrumenter.rb:23:in `instrument'
/opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/producer.rb:240:in `deliver_messages'
./myproject:38:in `block (2 levels) in run'
./myproject:33:in `each'
./myproject:33:in `block in run'
/opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/consumer.rb:312:in `block (3 levels) in each_batch'
/opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/instrumenter.rb:23:in `instrument'
/opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/instrumenter.rb:35:in `instrument'
/opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/consumer.rb:310:in `block (2 levels) in each_batch'
/opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/consumer.rb:292:in `each'
/opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/consumer.rb:292:in `block in each_batch'
/opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/consumer.rb:393:in `block in consumer_loop'
/opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/instrumenter.rb:23:in `instrument'
/opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/instrumenter.rb:35:in `instrument'
/opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/consumer.rb:392:in `consumer_loop'
/opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/consumer.rb:289:in `each_batch'
./myproject:31:in `run'
./myproject:64:in `block in start_worker'
I, [2018-10-02T08:57:27.539172 #18366]  INFO -- : Committing offsets: my.local.topic/0:32993049
D, [2018-10-02T08:57:27.539410 #18366] DEBUG -- : Sending offset_commit API request 604 to localhost:9092
D, [2018-10-02T08:57:27.540233 #18366] DEBUG -- : Waiting for response 604 from localhost:9092
D, [2018-10-02T08:57:27.540653 #18366] DEBUG -- : Received response 604 from localhost:9092
E, [2018-10-02T08:57:27.540831 #18366] ERROR -- : Error committing offsets: Kafka::UnknownMemberId
E, [2018-10-02T08:57:27.540961 #18366] ERROR -- : Retrying to make final offsets commit (3 attempts left)
I, [2018-10-02T08:57:27.641155 #18366]  INFO -- : Committing offsets: my.local.topic/0:32993049
D, [2018-10-02T08:57:27.641367 #18366] DEBUG -- : Sending offset_commit API request 605 to localhost:9092
D, [2018-10-02T08:57:27.641606 #18366] DEBUG -- : Waiting for response 605 from localhost:9092
D, [2018-10-02T08:57:27.642350 #18366] DEBUG -- : Received response 605 from localhost:9092
E, [2018-10-02T08:57:27.642550 #18366] ERROR -- : Error committing offsets: Kafka::UnknownMemberId
E, [2018-10-02T08:57:27.642623 #18366] ERROR -- : Retrying to make final offsets commit (2 attempts left)
I, [2018-10-02T08:57:27.742746 #18366]  INFO -- : Committing offsets: my.local.topic/0:32993049
D, [2018-10-02T08:57:27.742986 #18366] DEBUG -- : Sending offset_commit API request 606 to localhost:9092
D, [2018-10-02T08:57:27.743341 #18366] DEBUG -- : Waiting for response 606 from localhost:9092
D, [2018-10-02T08:57:27.744074 #18366] DEBUG -- : Received response 606 from localhost:9092
E, [2018-10-02T08:57:27.744273 #18366] ERROR -- : Error committing offsets: Kafka::UnknownMemberId
E, [2018-10-02T08:57:27.744346 #18366] ERROR -- : Retrying to make final offsets commit (1 attempts left)
I, [2018-10-02T08:57:27.844537 #18366]  INFO -- : Committing offsets: my.local.topic/0:32993049
D, [2018-10-02T08:57:27.844771 #18366] DEBUG -- : Sending offset_commit API request 607 to localhost:9092
D, [2018-10-02T08:57:27.845012 #18366] DEBUG -- : Waiting for response 607 from localhost:9092
D, [2018-10-02T08:57:27.845779 #18366] DEBUG -- : Received response 607 from localhost:9092
E, [2018-10-02T08:57:27.846002 #18366] ERROR -- : Error committing offsets: Kafka::UnknownMemberId
I, [2018-10-02T08:57:27.846189 #18366]  INFO -- : Leaving group `myproject`
D, [2018-10-02T08:57:27.846304 #18366] DEBUG -- : Sending leave_group API request 608 to localhost:9092
D, [2018-10-02T08:57:27.846515 #18366] DEBUG -- : Waiting for response 608 from localhost:9092
D, [2018-10-02T08:57:27.847111 #18366] DEBUG -- : Received response 608 from localhost:9092
W, [2018-10-02T08:57:27.847645 #18366]  WARN -- : Restarting crashed worker logship: Kafka::ProcessingError::Kafka::ProcessingError
D, [2018-10-02T08:57:27.847773 #18366] DEBUG -- : /opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/consumer.rb:321:in `rescue in block (3 levels) in each_batch'
D, [2018-10-02T08:57:27.847813 #18366] DEBUG -- : /opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/consumer.rb:324:in `block (3 levels) in each_batch'
D, [2018-10-02T08:57:27.847850 #18366] DEBUG -- : /opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/instrumenter.rb:23:in `instrument'
D, [2018-10-02T08:57:27.847887 #18366] DEBUG -- : /opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/instrumenter.rb:35:in `instrument'
D, [2018-10-02T08:57:27.848036 #18366] DEBUG -- : /opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/consumer.rb:310:in `block (2 levels) in each_batch'
D, [2018-10-02T08:57:27.848074 #18366] DEBUG -- : /opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/consumer.rb:292:in `each'
D, [2018-10-02T08:57:27.848110 #18366] DEBUG -- : /opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/consumer.rb:292:in `block in each_batch'
D, [2018-10-02T08:57:27.848148 #18366] DEBUG -- : /opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/consumer.rb:393:in `block in consumer_loop'
D, [2018-10-02T08:57:27.848184 #18366] DEBUG -- : /opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/instrumenter.rb:23:in `instrument'
D, [2018-10-02T08:57:27.848222 #18366] DEBUG -- : /opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/instrumenter.rb:35:in `instrument'
D, [2018-10-02T08:57:27.848258 #18366] DEBUG -- : /opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/consumer.rb:392:in `consumer_loop'
D, [2018-10-02T08:57:27.848295 #18366] DEBUG -- : /opt/myproject/vendor/bundler/ruby/2.3.0/gems/ruby-kafka-0.7.1/lib/kafka/consumer.rb:289:in `each_batch'
D, [2018-10-02T08:57:27.848331 #18366] DEBUG -- : ./myproject:31:in `run'
D, [2018-10-02T08:57:27.848367 #18366] DEBUG -- : ./myproject:64:in `block in start_worker'
D, [2018-10-02T08:57:28.063821 #18366] DEBUG -- : Handling fetcher command: stop


Product Browser not working after refresh and category/product was selected

Expected Behaviour
After refreshing the product browser when a category or product was selected, results in a broken product Browser. I am expecting to stil see all catalogs and the parent categories.
Actual Behaviour
After refreshing the product browser when a category or product was selected, results in a broken product Browser.I want to provide a link to a category or product, which will not render the catalogs anymore.
Instead the selected category or products becomes the root of the Product Browser and I cannot switch to another catalog anymore.
Steps to Reproduce

Open http://localhost:4502/libs/commerce/gui/content/products.html/var/commerce/products
Select a Catalog, a category and then a sub-category. This still works fine
Now refresh the Product browser or use the url in the address bar and open it in a new tab
Now the catalogs and categories are gone, and I only see the product. On top I also get an error message in the 'left pane'

Platform and Version
AEM 6.4.5 & CIF 0.9.0
(On CIF 0.8.0 we don't have this issue)
Sample Code that illustrates the problem

You should rather check the logs of your AEM instance and check any error message to find out why the loading of the categories has failed. But this is probably not a bug related to the scenario you describe, this works perfectly fine for me.


Add .gitignore for the Clean programming language

Clean is a lazy functional programming language developed on the Radboud University. In the development version Clean is bundled with the iTasks environment.

Homepage clean: http://clean.cs.ru.nl
Homepage iTasks: http://clean.cs.ru.nl/ITasks

Definitely seconded. Since GitHub already recognises *.icl and *.dcl as Clean files, this is a very useful addition.

Yes, this would be very useful.

Great addition.

This is superseeded by PR #2303


home page images aren't loaded for me

I'm not seeing any images on the home page, which makes it impossible to see the options in the main body. I can send you a screenshot.

working in Chrome, not in FF -- for me, it's all all images, including the logo in the header that aren't displaying.

I just opened in FF and looks same. your cache?
Chris Mathews
On Wed, Sep 14, 2016 1:02 PM, Jason Hibbets<EMAIL_ADDRESS>wrote:
working in Chrome, not in FF -- for me, it's all all images, including the logo
in the header that aren't displaying.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub, or mute the thread.

I cleared cache and did a shift+reload. same results. I'll check later. the images were just there, so this might be a problem with my browser

I cleared cache and restarted my browser, and I'm getting the following CSS error:
"JQMIGRATE: Migrate is installed, version 1.4.1" jquery-migrate.min.js:2:541
Use of getUserData() or setUserData() is deprecated.  Use WeakMap or element.dataset instead. BackgroundFramework.js:504:0
Use of getPreventDefault() is deprecated.  Use defaultPrevented instead. BackgroundFramework.js line 470 > eval:3:0
no element found

disabled server side caching. check it outplease
Chris Mathews
On Wed, Sep 14, 2016 1:18 PM, Jason Hibbets<EMAIL_ADDRESS>wrote:
I cleared cache and restarted my browser, and I'm getting the following CSS
error:
"JQMIGRATE: Migrate is installed, version 1.4.1" jquery-migrate.min.js:2:541
Use of getUserData() or setUserData() is deprecated. Use WeakMap or
element.dataset instead. BackgroundFramework.js:504:0
Use of getPreventDefault() is deprecated. Use defaultPrevented instead.
BackgroundFramework.js line 470 > eval:3:0
no element found
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub, or mute the thread.

Can't duplicate

Maybe we could contact the theme people for support?
Here is a new error I got after updating the site tonight:
downloadable font: OS/2: bad linegap: -32 (font-family: "kt-icon" style:normal weight:normal stretch:normal src index:1) source: http://citycampnc.com/wp-content/themes/pinnacle_premium/assets/css/fonts/kt-icon.woff?a-vp28cu pinnacle.css:1:78045
Use of getUserData() or setUserData() is deprecated.  Use WeakMap or element.dataset instead. BackgroundFramework.js:504:0
no element found mod_pagespeed_beacon:1:1
Use of getPreventDefault() is deprecated.  Use defaultPrevented instead.

Just tested at home, I think it's just me.

Ha. the problem only happens on citycampnc.com, when I switch to .org, the homepage is fine.

The theme people seem to say this is related to the cached copy of a font.
Seems to be isolated to a single users browser. Not wide spread. Tracy suggested
turning it and back on again.
Chris Mathews
On Wed, Sep 14, 2016 10:13 PM, Jason Hibbets<EMAIL_ADDRESS>wrote:
Ha. the problem only happens on citycampnc.com, when I switch to .org, the
homepage is fine.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub, or mute the thread.


Ollama: 500 error on Larger Models

What is the issue?
Whenever I try to run a model greater than the 7b or 8b, I get the following error. HOWEVER, any of the regular ones that are 7b and 8b run just fine.
Ollama: 500, message='Internal Server Error', url=URL('http://localhost:11434/api/chat')

Running Ubuntu Server 24.04
Running through docker
i7 2.1GHz
64GB RAM
GeForce RTX 4060 Ti 16GB

Any assistance would be appreciated
OS
Linux
GPU
Nvidia
CPU
Intel
Ollama version
No response

Server logs may make it easier to diagnose the issue.

Running Ollama through single docker: docker run -d -p 3000:8080 --gpus=all -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:ollama
However, I tried it with installing ollama as well and then just launching Docker instance with Openweb UI.. same thing

Server logs may make it easier to diagnose the issue.

How would I get this since its a docker container? Thanks in advance

docker logs

Log output below. What's odd is that this is running on <IP_ADDRESS>, yet it references another machine (in the logs below) with a .17 IP that is also running ollama with openweb UI....
INFO  [apps.ollama.main] url: http://localhost:11434
time=2024-07-23T21:07:22.397Z level=INFO source=sched.go:738 msg="new model will fit in available VRAM in single GPU, loading" model=/root/.ollama/models/blobs/sha256-b559938ab7a0392fc9ea9675b82280f2a15669ec3e0e0fc491c9cb0a7681cf94 gpu=GPU-02cae464-d29e-31cd-cd63-22666e511c22 parallel=4 available=16604921856 required="8.7 GiB"
time=2024-07-23T21:07:22.397Z level=INFO source=memory.go:309 msg="offload to cuda" layers.requested=-1 layers.model=41 layers.offload=41 layers.split="" memory.available="[15.5 GiB]" memory.required.full="8.7 GiB" memory.required.partial="8.7 GiB" memory.required.kv="1.2 GiB" memory.required.allocations="[8.7 GiB]" memory.weights.total="7.0 GiB" memory.weights.repeating="6.5 GiB" memory.weights.nonrepeating="525.0 MiB" memory.graph.full="568.0 MiB" memory.graph.partial="801.0 MiB"
time=2024-07-23T21:07:22.398Z level=INFO source=server.go:375 msg="starting llama server" cmd="/tmp/ollama3862763016/runners/cuda_v11/ollama_llama_server --model /root/.ollama/models/blobs/sha256-b559938ab7a0392fc9ea9675b82280f2a15669ec3e0e0fc491c9cb0a7681cf94 --ctx-size 8192 --batch-size 512 --embedding --log-disable --n-gpu-layers 41 --parallel 4 --port 40397"
time=2024-07-23T21:07:22.398Z level=INFO source=sched.go:474 msg="loaded runners" count=1
time=2024-07-23T21:07:22.398Z level=INFO source=server.go:563 msg="waiting for llama runner to start responding"
time=2024-07-23T21:07:22.398Z level=INFO source=server.go:604 msg="waiting for server to become available" status="llm server error"
INFO [main] build info | build=1 commit="a8db2a9" tid="123403186212864" timestamp=1721768842
INFO [main] system info | n_threads=8 n_threads_batch=-1 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 0 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 0 | " tid="123403186212864" timestamp=1721768842 total_threads=24
INFO [main] HTTP server listening | hostname="<IP_ADDRESS>" n_threads_http="23" port="40397" tid="123403186212864" timestamp=1721768842
llama_model_loader: loaded meta data with 35 key-value pairs and 363 tensors from /root/.ollama/models/blobs/sha256-b559938ab7a0392fc9ea9675b82280f2a15669ec3e0e0fc491c9cb0a7681cf94 (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = llama
llama_model_loader: - kv   1:                               general.type str              = model
llama_model_loader: - kv   2:                               general.name str              = Mistral Nemo Instruct 2407
llama_model_loader: - kv   3:                            general.version str              = 2407
llama_model_loader: - kv   4:                           general.finetune str              = Instruct
llama_model_loader: - kv   5:                           general.basename str              = Mistral-Nemo
llama_model_loader: - kv   6:                         general.size_label str              = 12B
llama_model_loader: - kv   7:                            general.license str              = apache-2.0
llama_model_loader: - kv   8:                          general.languages arr[str,9]       = ["en", "fr", "de", "es", "it", "pt", ...
llama_model_loader: - kv   9:                          llama.block_count u32              = 40
llama_model_loader: - kv  10:                       llama.context_length u32              = 1024000
llama_model_loader: - kv  11:                     llama.embedding_length u32              = 5120
llama_model_loader: - kv  12:                  llama.feed_forward_length u32              = 14336
llama_model_loader: - kv  13:                 llama.attention.head_count u32              = 32
llama_model_loader: - kv  14:              llama.attention.head_count_kv u32              = 8
llama_model_loader: - kv  15:                       llama.rope.freq_base f32              = 1000000.000000
llama_model_loader: - kv  16:     llama.attention.layer_norm_rms_epsilon f32              = 0.000010
llama_model_loader: - kv  17:                 llama.attention.key_length u32              = 128
llama_model_loader: - kv  18:               llama.attention.value_length u32              = 128
llama_model_loader: - kv  19:                          general.file_type u32              = 2
llama_model_loader: - kv  20:                           llama.vocab_size u32              = 131072
llama_model_loader: - kv  21:                 llama.rope.dimension_count u32              = 128
llama_model_loader: - kv  22:            tokenizer.ggml.add_space_prefix bool             = false
llama_model_loader: - kv  23:                       tokenizer.ggml.model str              = gpt2
llama_model_loader: - kv  24:                         tokenizer.ggml.pre str              = tekken
llama_model_loader: - kv  25:                      tokenizer.ggml.tokens arr[str,131072]  = ["", "", "", "[INST]", "[...
llama_model_loader: - kv  26:                  tokenizer.ggml.token_type arr[i32,131072]  = [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ...
llama_model_loader: - kv  27:                      tokenizer.ggml.merges arr[str,269443]  = ["Ġ Ġ", "Ġ t", "e r", "i n", "Ġ �...
llama_model_loader: - kv  28:                tokenizer.ggml.bos_token_id u32              = 1
llama_model_loader: - kv  29:                tokenizer.ggml.eos_token_id u32              = 2
llama_model_loader: - kv  30:            tokenizer.ggml.unknown_token_id u32              = 0
llama_model_loader: - kv  31:               tokenizer.ggml.add_bos_token bool             = true
llama_model_loader: - kv  32:               tokenizer.ggml.add_eos_token bool             = false
llama_model_loader: - kv  33:                    tokenizer.chat_template str              = {%- if messages[0]['role'] == 'system...
llama_model_loader: - kv  34:               general.quantization_version u32              = 2
llama_model_loader: - type  f32:   81 tensors
llama_model_loader: - type q4_0:  281 tensors
llama_model_loader: - type q6_K:    1 tensors
llm_load_vocab: missing or unrecognized pre-tokenizer type, using: 'default'
time=2024-07-23T21:07:22.650Z level=INFO source=server.go:604 msg="waiting for server to become available" status="llm server loading model"
llm_load_vocab: special tokens cache size = 1000
llm_load_vocab: token to piece cache size = 0.8498 MB
llm_load_print_meta: format           = GGUF V3 (latest)
llm_load_print_meta: arch             = llama
llm_load_print_meta: vocab type       = BPE
llm_load_print_meta: n_vocab          = 131072
llm_load_print_meta: n_merges         = 269443
llm_load_print_meta: vocab_only       = 0
llm_load_print_meta: n_ctx_train      = 1024000
llm_load_print_meta: n_embd           = 5120
llm_load_print_meta: n_layer          = 40
llm_load_print_meta: n_head           = 32
llm_load_print_meta: n_head_kv        = 8
llm_load_print_meta: n_rot            = 128
llm_load_print_meta: n_swa            = 0
llm_load_print_meta: n_embd_head_k    = 128
llm_load_print_meta: n_embd_head_v    = 128
llm_load_print_meta: n_gqa            = 4
llm_load_print_meta: n_embd_k_gqa     = 1024
llm_load_print_meta: n_embd_v_gqa     = 1024
llm_load_print_meta: f_norm_eps       = 0.0e+00
llm_load_print_meta: f_norm_rms_eps   = 1.0e-05
llm_load_print_meta: f_clamp_kqv      = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: f_logit_scale    = 0.0e+00
llm_load_print_meta: n_ff             = 14336
llm_load_print_meta: n_expert         = 0
llm_load_print_meta: n_expert_used    = 0
llm_load_print_meta: causal attn      = 1
llm_load_print_meta: pooling type     = 0
llm_load_print_meta: rope type        = 0
llm_load_print_meta: rope scaling     = linear
llm_load_print_meta: freq_base_train  = 1000000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: n_ctx_orig_yarn  = 1024000
llm_load_print_meta: rope_finetuned   = unknown
llm_load_print_meta: ssm_d_conv       = 0
llm_load_print_meta: ssm_d_inner      = 0
llm_load_print_meta: ssm_d_state      = 0
llm_load_print_meta: ssm_dt_rank      = 0
llm_load_print_meta: model type       = 13B
llm_load_print_meta: model ftype      = Q4_0
llm_load_print_meta: model params     = 12.25 B
llm_load_print_meta: model size       = 6.58 GiB (4.61 BPW)
llm_load_print_meta: general.name     = Mistral Nemo Instruct 2407
llm_load_print_meta: BOS token        = 1 ''
llm_load_print_meta: EOS token        = 2 ''
llm_load_print_meta: UNK token        = 0 ''
llm_load_print_meta: LF token         = 1196 'Ä'
llm_load_print_meta: max token length = 150
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    yes
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
Device 0: NVIDIA GeForce RTX 4060 Ti, compute capability 8.9, VMM: yes
llm_load_tensors: ggml ctx size =    0.34 MiB
llama_model_load: error loading model: check_tensor_dims: tensor 'blk.0.attn_q.weight' has wrong shape; expected  5120,  5120, got  5120,  4096,     1,     1
llama_load_model_from_file: exception loading model
terminate called after throwing an instance of 'std::runtime_error'
what():  check_tensor_dims: tensor 'blk.0.attn_q.weight' has wrong shape; expected  5120,  5120, got  5120,  4096,     1,     1
time=2024-07-23T21:07:23.152Z level=ERROR source=sched.go:480 msg="error loading llama server" error="llama runner process has terminated: signal: aborted (core dumped) error loading model: check_tensor_dims: tensor 'blk.0.attn_q.weight' has wrong shape; expected  5120,  5120, got  5120,  4096,     1,     1\nllama_load_model_from_file: exception loading model"
[GIN] 2024/07/23 - 21:07:23 | 500 |  901.741973ms |       <IP_ADDRESS> | POST     "/api/chat"
INFO:     <IP_ADDRESS>:53773 - "POST /ollama/api/chat HTTP/1.1" 500 Internal Server Error
ERROR [asyncio] Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7afabc379390>
INFO:     <IP_ADDRESS>:53773 - "GET /api/v1/chats/ HTTP/1.1" 200 OK
time=2024-07-23T21:07:28.273Z level=WARN source=sched.go:671 msg="gpu VRAM usage didn't recover within timeout" seconds=5.12139152 model=/root/.ollama/models/blobs/sha256-b559938ab7a0392fc9ea9675b82280f2a15669ec3e0e0fc491c9cb0a7681cf94
time=2024-07-23T21:07:28.523Z level=WARN source=sched.go:671 msg="gpu VRAM usage didn't recover within timeout" seconds=5.371245575 model=/root/.ollama/models/blobs/sha256-b559938ab7a0392fc9ea9675b82280f2a15669ec3e0e0fc491c9cb0a7681cf94
time=2024-07-23T21:07:28.773Z level=WARN source=sched.go:671 msg="gpu VRAM usage didn't recover within timeout" seconds=5.621260718 model=/root/.ollama/models/blobs/sha256-b559938ab7a0392fc9ea9675b82280f2a15669ec3e0e0fc491c9cb0a7681cf94
INFO:     <IP_ADDRESS>:46384 - "GET /health HTTP/1.1" 200 OK
INFO:     <IP_ADDRESS>:55030 - "GET /health HTTP/1.1" 200 OK
INFO:     <IP_ADDRESS>:53856 - "GET /health HTTP/1.1" 200 OK
INFO:     <IP_ADDRESS>:46666 - "GET /health HTTP/1.1" 200 OK
INFO:     <IP_ADDRESS>:45528 - "GET /health HTTP/1.1" 200 OK
INFO:     <IP_ADDRESS>:56908 - "GET /health HTTP/1.1" 200 OK
INFO:     <IP_ADDRESS>:47616 - "GET /health HTTP/1.1" 200 OK
INFO  [apps.ollama.main] url: http://localhost:11434
time=2024-07-23T21:11:01.686Z level=INFO source=sched.go:738 msg="new model will fit in available VRAM in single GPU, loading" model=/root/.ollama/models/blobs/sha256-b559938ab7a0392fc9ea9675b82280f2a15669ec3e0e0fc491c9cb0a7681cf94 gpu=GPU-02cae464-d29e-31cd-cd63-22666e511c22 parallel=4 available=16604921856 required="8.7 GiB"
time=2024-07-23T21:11:01.686Z level=INFO source=memory.go:309 msg="offload to cuda" layers.requested=-1 layers.model=41 layers.offload=41 layers.split="" memory.available="[15.5 GiB]" memory.required.full="8.7 GiB" memory.required.partial="8.7 GiB" memory.required.kv="1.2 GiB" memory.required.allocations="[8.7 GiB]" memory.weights.total="7.0 GiB" memory.weights.repeating="6.5 GiB" memory.weights.nonrepeating="525.0 MiB" memory.graph.full="568.0 MiB" memory.graph.partial="801.0 MiB"
time=2024-07-23T21:11:01.687Z level=INFO source=server.go:375 msg="starting llama server" cmd="/tmp/ollama3862763016/runners/cuda_v11/ollama_llama_server --model /root/.ollama/models/blobs/sha256-b559938ab7a0392fc9ea9675b82280f2a15669ec3e0e0fc491c9cb0a7681cf94 --ctx-size 8192 --batch-size 512 --embedding --log-disable --n-gpu-layers 41 --parallel 4 --port 42509"
time=2024-07-23T21:11:01.687Z level=INFO source=sched.go:474 msg="loaded runners" count=1
time=2024-07-23T21:11:01.687Z level=INFO source=server.go:563 msg="waiting for llama runner to start responding"
time=2024-07-23T21:11:01.687Z level=INFO source=server.go:604 msg="waiting for server to become available" status="llm server error"
INFO [main] build info | build=1 commit="a8db2a9" tid="126845667917824" timestamp=1721769061
INFO [main] system info | n_threads=8 n_threads_batch=-1 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 0 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 0 | " tid="126845667917824" timestamp=1721769061 total_threads=24
INFO [main] HTTP server listening | hostname="<IP_ADDRESS>" n_threads_http="23" port="42509" tid="126845667917824" timestamp=1721769061
llama_model_loader: loaded meta data with 35 key-value pairs and 363 tensors from /root/.ollama/models/blobs/sha256-b559938ab7a0392fc9ea9675b82280f2a15669ec3e0e0fc491c9cb0a7681cf94 (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = llama
llama_model_loader: - kv   1:                               general.type str              = model
llama_model_loader: - kv   2:                               general.name str              = Mistral Nemo Instruct 2407
llama_model_loader: - kv   3:                            general.version str              = 2407
llama_model_loader: - kv   4:                           general.finetune str              = Instruct
llama_model_loader: - kv   5:                           general.basename str              = Mistral-Nemo
llama_model_loader: - kv   6:                         general.size_label str              = 12B
llama_model_loader: - kv   7:                            general.license str              = apache-2.0
llama_model_loader: - kv   8:                          general.languages arr[str,9]       = ["en", "fr", "de", "es", "it", "pt", ...
llama_model_loader: - kv   9:                          llama.block_count u32              = 40
llama_model_loader: - kv  10:                       llama.context_length u32              = 1024000
llama_model_loader: - kv  11:                     llama.embedding_length u32              = 5120
llama_model_loader: - kv  12:                  llama.feed_forward_length u32              = 14336
llama_model_loader: - kv  13:                 llama.attention.head_count u32              = 32
llama_model_loader: - kv  14:              llama.attention.head_count_kv u32              = 8
llama_model_loader: - kv  15:                       llama.rope.freq_base f32              = 1000000.000000
llama_model_loader: - kv  16:     llama.attention.layer_norm_rms_epsilon f32              = 0.000010
llama_model_loader: - kv  17:                 llama.attention.key_length u32              = 128
llama_model_loader: - kv  18:               llama.attention.value_length u32              = 128
llama_model_loader: - kv  19:                          general.file_type u32              = 2
llama_model_loader: - kv  20:                           llama.vocab_size u32              = 131072
llama_model_loader: - kv  21:                 llama.rope.dimension_count u32              = 128
llama_model_loader: - kv  22:            tokenizer.ggml.add_space_prefix bool             = false
llama_model_loader: - kv  23:                       tokenizer.ggml.model str              = gpt2
llama_model_loader: - kv  24:                         tokenizer.ggml.pre str              = tekken
llama_model_loader: - kv  25:                      tokenizer.ggml.tokens arr[str,131072]  = ["", "", "", "[INST]", "[...
llama_model_loader: - kv  26:                  tokenizer.ggml.token_type arr[i32,131072]  = [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ...
llama_model_loader: - kv  27:                      tokenizer.ggml.merges arr[str,269443]  = ["Ġ Ġ", "Ġ t", "e r", "i n", "Ġ �...
llama_model_loader: - kv  28:                tokenizer.ggml.bos_token_id u32              = 1
llama_model_loader: - kv  29:                tokenizer.ggml.eos_token_id u32              = 2
llama_model_loader: - kv  30:            tokenizer.ggml.unknown_token_id u32              = 0
llama_model_loader: - kv  31:               tokenizer.ggml.add_bos_token bool             = true
llama_model_loader: - kv  32:               tokenizer.ggml.add_eos_token bool             = false
llama_model_loader: - kv  33:                    tokenizer.chat_template str              = {%- if messages[0]['role'] == 'system...
llama_model_loader: - kv  34:               general.quantization_version u32              = 2
llama_model_loader: - type  f32:   81 tensors
llama_model_loader: - type q4_0:  281 tensors
llama_model_loader: - type q6_K:    1 tensors
llm_load_vocab: missing or unrecognized pre-tokenizer type, using: 'default'
llm_load_vocab: special tokens cache size = 1000
time=2024-07-23T21:11:01.938Z level=INFO source=server.go:604 msg="waiting for server to become available" status="llm server loading model"
llm_load_vocab: token to piece cache size = 0.8498 MB
llm_load_print_meta: format           = GGUF V3 (latest)
llm_load_print_meta: arch             = llama
llm_load_print_meta: vocab type       = BPE
llm_load_print_meta: n_vocab          = 131072
llm_load_print_meta: n_merges         = 269443
llm_load_print_meta: vocab_only       = 0
llm_load_print_meta: n_ctx_train      = 1024000
llm_load_print_meta: n_embd           = 5120
llm_load_print_meta: n_layer          = 40
llm_load_print_meta: n_head           = 32
llm_load_print_meta: n_head_kv        = 8
llm_load_print_meta: n_rot            = 128
llm_load_print_meta: n_swa            = 0
llm_load_print_meta: n_embd_head_k    = 128
llm_load_print_meta: n_embd_head_v    = 128
llm_load_print_meta: n_gqa            = 4
llm_load_print_meta: n_embd_k_gqa     = 1024
llm_load_print_meta: n_embd_v_gqa     = 1024
llm_load_print_meta: f_norm_eps       = 0.0e+00
llm_load_print_meta: f_norm_rms_eps   = 1.0e-05
llm_load_print_meta: f_clamp_kqv      = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: f_logit_scale    = 0.0e+00
llm_load_print_meta: n_ff             = 14336
llm_load_print_meta: n_expert         = 0
llm_load_print_meta: n_expert_used    = 0
llm_load_print_meta: causal attn      = 1
llm_load_print_meta: pooling type     = 0
llm_load_print_meta: rope type        = 0
llm_load_print_meta: rope scaling     = linear
llm_load_print_meta: freq_base_train  = 1000000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: n_ctx_orig_yarn  = 1024000
llm_load_print_meta: rope_finetuned   = unknown
llm_load_print_meta: ssm_d_conv       = 0
llm_load_print_meta: ssm_d_inner      = 0
llm_load_print_meta: ssm_d_state      = 0
llm_load_print_meta: ssm_dt_rank      = 0
llm_load_print_meta: model type       = 13B
llm_load_print_meta: model ftype      = Q4_0
llm_load_print_meta: model params     = 12.25 B
llm_load_print_meta: model size       = 6.58 GiB (4.61 BPW)
llm_load_print_meta: general.name     = Mistral Nemo Instruct 2407
llm_load_print_meta: BOS token        = 1 ''
llm_load_print_meta: EOS token        = 2 ''
llm_load_print_meta: UNK token        = 0 ''
llm_load_print_meta: LF token         = 1196 'Ä'
llm_load_print_meta: max token length = 150
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    yes
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
Device 0: NVIDIA GeForce RTX 4060 Ti, compute capability 8.9, VMM: yes
llm_load_tensors: ggml ctx size =    0.34 MiB
llama_model_load: error loading model: check_tensor_dims: tensor 'blk.0.attn_q.weight' has wrong shape; expected  5120,  5120, got  5120,  4096,     1,     1
llama_load_model_from_file: exception loading model
terminate called after throwing an instance of 'std::runtime_error'
what():  check_tensor_dims: tensor 'blk.0.attn_q.weight' has wrong shape; expected  5120,  5120, got  5120,  4096,     1,     1
time=2024-07-23T21:11:02.439Z level=ERROR source=sched.go:480 msg="error loading llama server" error="llama runner process has terminated: signal: aborted (core dumped) error loading model: check_tensor_dims: tensor 'blk.0.attn_q.weight' has wrong shape; expected  5120,  5120, got  5120,  4096,     1,     1\nllama_load_model_from_file: exception loading model"
[GIN] 2024/07/23 - 21:11:02 | 500 |  904.787254ms |       <IP_ADDRESS> | POST     "/api/chat"
INFO:     <IP_ADDRESS>:53945 - "POST /ollama/api/chat HTTP/1.1" 500 Internal Server Error
INFO:     <IP_ADDRESS>:53945 - "GET /api/v1/chats/ HTTP/1.1" 200 OK
time=2024-07-23T21:11:07.570Z level=WARN source=sched.go:671 msg="gpu VRAM usage didn't recover within timeout" seconds=5.13067364 model=/root/.ollama/models/blobs/sha256-b559938ab7a0392fc9ea9675b82280f2a15669ec3e0e0fc491c9cb0a7681cf94
time=2024-07-23T21:11:07.819Z level=WARN source=sched.go:671 msg="gpu VRAM usage didn't recover within timeout" seconds=5.379864756 model=/root/.ollama/models/blobs/sha256-b559938ab7a0392fc9ea9675b82280f2a15669ec3e0e0fc491c9cb0a7681cf94
time=2024-07-23T21:11:08.069Z level=WARN source=sched.go:671 msg="gpu VRAM usage didn't recover within timeout" seconds=5.630042941 model=/root/.ollama/models/blobs/sha256-b559938ab7a0392fc9ea9675b82280f2a15669ec3e0e0fc491c9cb0a7681cf94
['x2BJHVzmH2bs-VVMAAAD']
INFO:     <IP_ADDRESS>:39704 - "GET /health HTTP/1.1" 200 OK
ERROR [asyncio] Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7afabbe97190>
INFO:     <IP_ADDRESS>:58774 - "GET /health HTTP/1.1" 200 OK

llama_model_load: error loading model: check_tensor_dims: tensor 'blk.0.attn_q.weight' has wrong shape; expected 5120, 5120, got 5120, 4096, 1, 1

You are trying to run a model which is not supported by your version of ollama.  The start of the logs that identifies the version is not included, but it's probably less than 0.2.8.  You can try upgrading to the most recent version of ollama, but Mistral-Nemo support has only just been added and so it may not perform as well as it could until all of the bugs are ironed out.

Thank you very much. I will investigate that
From: frob @.>
Date: Tuesday, July 23, 2024 at 5:51 PM
To: ollama/ollama @.>
Cc: nicholhai @.>, Author @.>
Subject: Re: [ollama/ollama] Ollama: 500 error on Larger Models (Issue #5892)
llama_model_load: error loading model: check_tensor_dims: tensor 'blk.0.attn_q.weight' has wrong shape; expected 5120, 5120, got 5120, 4096, 1, 1
You are trying to run a model which is not supported by your version of ollama. The start of the logs that identifies the version is not included, but it's probably less than 0.2.8. You can try upgrading to the most recent version of ollama, but Mistral-Nemo support has only just been added and so it may not perform as well as it could until all of the bugs are ironed out.
—
Reply to this email directly, view it on GitHubhttps://github.com/ollama/ollama/issues/5892#issuecomment-2246376871, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AW6WDRDGHAVK4MHEEXDSK5TZN3F6HAVCNFSM6AAAAABLLF7FN2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBWGM3TMOBXGE.
You are receiving this because you authored the thread.Message ID: @.***>

Can you tell me how you've started the docker container? The underlying images used for the creation of the docker containers will not be pulled automatically. You need to manually issue a command such as this: docker pull ollama/ollama:rocm (I use the rocm version because I run ollama on an AMD card, that might be different for you), then remove the old container and create a new one. At least that's what works for me.
Also about the IP address being different in the logs: That's likely because it's showing the internal docker IP assigned to the container ollama's running in.

docker run -d -p 3000:8080 --gpus=all -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:ollama

If you add --pull always it will always pull the newest version when you start.

I added the --pull always at the end:
sudo docker run -d -p 3000:8080 --gpus=all -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:ollama --pull always
Got the following error:
docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "--pull": executable file not found in $PATH: unknown

Put it before the name of the image being pulled:
docker run -d -p 3000:8080 --gpus=all -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart always  --pull always ghcr.io/open-webui/open-webui:ollama

Yes, I realized that.. I did it here:
sudo docker run -d -p 3000:8080 --gpus=all -v ollama:/root/.ollama --pull always -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:ollama
Thank you.

It started. However, the original error still persists.. I have it running all the large models on my Mac Studio, no issues and I followed the same installation guide from https://docs.openwebui.com/getting-started/
Not sure what the difference is. I have wiped the server and reinstalled this using all the methods on that page, 20 times over
ollama: Pulling from open-webui/open-webui
Digest: sha256:199f1d5e5bf5c6954af376af3738e1ff76aab4987677abd177d01e5b97b6ca2c
Status: Image is up to date for ghcr.io/open-webui/open-webui:ollama
c7aecdd05e00ef66a8fb0b31021e83e4b47a5e062dddbc8ca19376f698543100

The version of ollama in that container is 0.2.1.  You can wait until they update ghcr.io/open-webui/open-webui:ollama or run standalone ghcr.io/open-webui/open-webui with your own ollama instance.

The confusing part is that it all runs on my Mac Studio.... any ideas why or what is different? Any other suggestions for installing a whole new instance of ollama and openweb UI that will work with Ubuntu server? (or any other OS as I have this machine dedicated to this)

@nicholhai it looks like the container you're starting is the open-webui one. That is not ollama, but the web UI you use to chat with LLMs. Do you have another container running that is ollama?

@nicholhai it looks like the container you're starting is the open-webui one. That is not ollama, but the web UI you use to chat with LLMs. Do you have another container running that is ollama?

That is the "all-in-one" container that will run ollama and web UI according to the instructions on : https://docs.openwebui.com/getting-started/

Thanks for the clarification. I wasn't aware that there was such a thing.

Maybe try running ollama in a separate container then. The official ollama images are updated more frequently.

Maybe try running ollama in a separate container then. The official ollama images are updated more frequently.

I don't know how to do that, unfortunately. Any help would be appreciated as I am not as skilled in docker containers and how to reference one from another

Start the ollama container:
docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

Start the open-webui container:
docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Maybe try running ollama in a separate container then. The official ollama images are updated more frequently.

I don't know how to do that, unfortunately. Any help would be appreciated as I am not as skilled in docker containers and how to reference one from another

Sure. I'd first read this
https://hub.docker.com/r/ollama/ollama
It will explain how to run ollama using docker.
After that you should check the official open-webui docs: https://github.com/open-webui/open-webui?tab=readme-ov-file#installation-with-default-configuration
It explains the steps in order to use the local ollama container with the UI. I use the same setup myself and it's been working fine for a long time now. If you need any help feel free to ask.

It's curious that your Mac Studio works.  Server logs from that might be illuminating.

Start the ollama container:
docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

Start the open-webui container:
docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Thank you. Trying this now... how would I run this with GPU support? just add --gpus=all ?

 When I try to download a model after starting the above two, I get:
Download Cancelled
Open WebUI: Server Connection Error

Logs from both containers.

Openweb Container
client_session: <aiohttp.client.ClientSession object at 0x7104b17a7c50>
INFO:     <IP_ADDRESS>:51151 - "POST /ollama/api/pull/0 HTTP/1.1" 500 Internal Server Error
INFO  [apps.openai.main] get_all_models()
INFO  [apps.ollama.main] get_all_models()
ERROR [apps.ollama.main] Connection error: Cannot connect to host localhost:11434 ssl:default [Connection refused]
INFO:     <IP_ADDRESS>:49484 - "GET /health HTTP/1.1" 200 OK
Ollama Container
2024/07/24 11:21:23 routes.go:1100: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: OLLAMA_DEBUG:false OLLAMA_FLASH_ATTENTION:false OLLAMA_HOST:http://<IP_ADDRESS>:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/root/.ollama/models OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://<IP_ADDRESS> https://<IP_ADDRESS> http://<IP_ADDRESS>:* https://<IP_ADDRESS>:* http://<IP_ADDRESS> https://<IP_ADDRESS> http://<IP_ADDRESS>:* https://<IP_ADDRESS>:* app://* file://* tauri://*] OLLAMA_RUNNERS_DIR: OLLAMA_SCHED_SPREAD:false OLLAMA_TMPDIR: ROCR_VISIBLE_DEVICES:]"
time=2024-07-24T11:21:23.353Z level=INFO source=images.go:784 msg="total blobs: 44"
time=2024-07-24T11:21:23.354Z level=INFO source=images.go:791 msg="total unused blobs removed: 0"
time=2024-07-24T11:21:23.354Z level=INFO source=routes.go:1147 msg="Listening on [::]:11434 (version 0.2.8)"
time=2024-07-24T11:21:23.355Z level=INFO source=payload.go:30 msg="extracting embedded files" dir=/tmp/ollama1220858021/runners
time=2024-07-24T11:21:26.002Z level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cuda_v11 rocm_v60102 cpu cpu_avx cpu_avx2]"
time=2024-07-24T11:21:26.002Z level=INFO source=gpu.go:205 msg="looking for compatible GPUs"
time=2024-07-24T11:21:26.239Z level=INFO source=types.go:105 msg="inference compute" id=GPU-02cae464-d29e-31cd-cd63-22666e511c22 library=cuda compute=8.9 driver=12.5 name="NVIDIA GeForce RTX 4060 Ti" total="15.6 GiB" available="15.5 GiB"

The webui container tries to access ollama running on localhost. But since ollama is running in a separate container, you will have to explicitly set the hostname of ollama in the webui.

How would I get the hostname for the ollama container?

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -e OLLAMA_BASE_URL=http://host.docker.internal:11434 -v open-webui:/app/backend/data --name open-webui --restart always --pull always ghcr.io/open-webui/open-webui:main

I stopped both docker containers: sudo docker rm -f $(sudo docker ps -a -q)
Then I started them both again one by one (this time using the new one you just posted): Same error for
sudo docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
sudo docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -e OLLAMA_BASE_URL=http://host.docker.internal:11434 -v open-webui:/app/backend/data --name open-webui --restart always --pull always ghcr.io/open-webui/open-webui:main

Hmm, worked fine here.  Logs from new instances?

2024/07/24 11:54:17 routes.go:1100: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: OLLAMA_DEBUG:false OLLAMA_FLASH_ATTENTION:false OLLAMA_HOST:http://<IP_ADDRESS>:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/root/.ollama/models OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://<IP_ADDRESS> https://<IP_ADDRESS> http://<IP_ADDRESS>:* https://<IP_ADDRESS>:* http://<IP_ADDRESS> https://<IP_ADDRESS> http://<IP_ADDRESS>:* https://<IP_ADDRESS>:* app://* file://* tauri://*] OLLAMA_RUNNERS_DIR: OLLAMA_SCHED_SPREAD:false OLLAMA_TMPDIR: ROCR_VISIBLE_DEVICES:]"
time=2024-07-24T11:54:17.503Z level=INFO source=images.go:784 msg="total blobs: 44"
time=2024-07-24T11:54:17.503Z level=INFO source=images.go:791 msg="total unused blobs removed: 0"
time=2024-07-24T11:54:17.504Z level=INFO source=routes.go:1147 msg="Listening on [::]:11434 (version 0.2.8)"
time=2024-07-24T11:54:17.505Z level=INFO source=payload.go:30 msg="extracting embedded files" dir=/tmp/ollama3984924101/runners
time=2024-07-24T11:54:19.630Z level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2 cuda_v11 rocm_v60102]"
time=2024-07-24T11:54:19.630Z level=INFO source=gpu.go:205 msg="looking for compatible GPUs"
time=2024-07-24T11:54:19.863Z level=INFO source=types.go:105 msg="inference compute" id=GPU-02cae464-d29e-31cd-cd63-22666e511c22 library=cuda compute=8.9 driver=12.5 name="NVIDIA GeForce RTX 4060 Ti" total="15.6 GiB" available="15.5 GiB
INFO:     <IP_ADDRESS>:45242 - "GET /health HTTP/1.1" 200 OK
INFO  [apps.openai.main] get_all_models()
INFO  [apps.ollama.main] get_all_models()
ERROR [apps.ollama.main] Connection error: Cannot connect to host localhost:11434 ssl:default [Connection refused]
INFO:     <IP_ADDRESS>:44652 - "GET /health HTTP/1.1" 200 OK
INFO  [apps.openai.main] get_all_models()
INFO  [apps.ollama.main] get_all_models()
ERROR [apps.ollama.main] Connection error: Cannot connect to host localhost:11434 ssl:default [Connection refused]
INFO:     <IP_ADDRESS>:59950 - "GET /health HTTP/1.1" 200 OK

Can you try this:
To start ollama: docker run --restart always -d --device /dev/kfd --device /dev/dri -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
To start open-webui: docker run -d --network=host -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=http://<IP_ADDRESS>:11434 --name open-webui --restart always ghcr.io/open-webui/open-webui:main
These are the commands I am using.
NOTE: Make sure you change the image in the ollama command to suit your needs. I saw you have an nvidia card so you'll have to adjust the image in order to get GPU acceleration.

INFO:     <IP_ADDRESS>:52600 - "GET /ollama/config HTTP/1.1" 200 OK
INFO  [apps.openai.main] get_all_models()
INFO  [apps.ollama.main] get_all_models()
INFO  [apps.openai.main] get_all_models()
INFO  [apps.ollama.main] get_all_models()
ERROR [apps.ollama.main] Connection error: Cannot connect to host localhost:11434 ssl:default [Connection refused]
ERROR [apps.ollama.main] Connection error: Cannot connect to host localhost:11434 ssl:default [Connection refused]
INFO  [apps.ollama.main] get_all_models()
INFO  [apps.ollama.main] get_all_models()
ERROR [apps.ollama.main] Connection error: Cannot connect to host localhost:11434 ssl:default [Connection refused]
ERROR [apps.ollama.main] Connection error: Cannot connect to host localhost:11434 ssl:default [Connection refused]
ERROR [apps.ollama.main] Connection error: Cannot connect to host localhost:11434 ssl:default [Connection refused]

Still trying to connect to localhost.  Do docker stop open-webui ; docker rm open-webui and then run the docker run command for open-webui.

ollama image contains Nvidia support.

Also something you should try: After removing the containers, issue a docker image prune -a to ensure the old, possibly outdated, images are also deleted.
IIRC simply issuing docker rm -f does not remove the images of the containers.

Trying all the above now. Thank you so much. Appreciate all the support

Also something you should try: After removing the containers, issue a docker image prune -a to ensure the old, possibly outdated, images are also deleted.
IIRC simply issuing docker rm -f does not remove the images of the containers.

Whoa. that removed a whole whack of containers. Thanks @Cephra

sudo docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -e OLLAMA_BASE_URL=http://host.docker.internal:11434 -v open-webui:/app/backend/data --name open-webui --restart always --pull always ghcr.io/open-webui/open-webui:main

stopped all docker containers
removed them
pruned
started then both again

INFO:     <IP_ADDRESS>:53485 - "GET /ollama/config HTTP/1.1" 200 OK
INFO  [apps.openai.main] get_all_models()
INFO  [apps.ollama.main] get_all_models()
INFO  [apps.openai.main] get_all_models()
INFO  [apps.ollama.main] get_all_models()
ERROR [apps.ollama.main] Connection error: Cannot connect to host localhost:11434 ssl:default [Connection refused]
INFO  [apps.ollama.main] get_all_models()
ERROR [apps.ollama.main] Connection error: Cannot connect to host localhost:11434 ssl:default [Connection refused]
INFO  [apps.ollama.main] get_all_models()
ERROR [apps.ollama.main] Connection error: Cannot connect to host localhost:11434 ssl:default [Connection refused]
ERROR [apps.ollama.main] Connection error: Cannot connect to host localhost:11434 ssl:default [Connection refused]
ERROR [apps.ollama.main] Connection error: Cannot connect to host localhost:11434 ssl:default [Connection refused]
INFO:     <IP_ADDRESS>:53455 - "GET /ollama/api/version HTTP/1.1" 500 Internal Server Error
INFO:     <IP_ADDRESS>:53485 - "GET /ollama/urls HTTP/1.1" 200 OK

Can you try this:
To start ollama: docker run --restart always -d --device /dev/kfd --device /dev/dri -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
To start open-webui: docker run -d --network=host -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=http://<IP_ADDRESS>:11434 --name open-webui --restart always ghcr.io/open-webui/open-webui:main
These are the commands I am using.
NOTE: Make sure you change the image in the ollama command to suit your needs. I saw you have an nvidia card so you'll have to adjust the image in order to get GPU acceleration.

docker: Error response from daemon: error gathering device information while adding custom device "/dev/kfd": no such file or directory.
I am still learning docker ...

Can you try this:
To start ollama: docker run --restart always -d --device /dev/kfd --device /dev/dri -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
To start open-webui: docker run -d --network=host -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=http://<IP_ADDRESS>:11434 --name open-webui --restart always ghcr.io/open-webui/open-webui:main
These are the commands I am using.
NOTE: Make sure you change the image in the ollama command to suit your needs. I saw you have an nvidia card so you'll have to adjust the image in order to get GPU acceleration.

docker: Error response from daemon: error gathering device information while adding custom device "/dev/kfd": no such file or directory.
I am still learning docker ...

Oh okay.. maybe those switches are Linux exclusive. For ollama try this command instead: docker run --restart always --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

Nope. the above won't even launch the localhost:3000

You have an nvidia card, --gpus all is all you need.

Maybe there is saved state.
docker stop open-webui
docker rm open-webui
docker volume rm open-webui
docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -e OLLAMA_BASE_URL=http://host.docker.internal:11434 -v open-webui:/app/backend/data --name open-webui --restart always --pull always ghcr.io/open-webui/open-webui:main

I'm pretty sure @rick-github is right! There's probably still a volume with runtime config for open-webui.

Nope. the above won't even launch the localhost:3000

You mean the container isn't even started? Do you get any error message or something?

Nope. the above won't even launch the localhost:3000

You mean the container isn't even started? Do you get any error message or something?

It started. Just went to "Page not found..."

The models are stored in the ollama container, the user info is stored in the open-webui container, which was deleted with the docker volume rm open-webui command.

Maybe there is saved state.
docker stop open-webui
docker rm open-webui
docker volume rm open-webui
docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -e OLLAMA_BASE_URL=http://host.docker.internal:11434 -v open-webui:/app/backend/data --name open-webui --restart always --pull always ghcr.io/open-webui/open-webui:main

We have a winner! I think the "docker volume rm open-webui" did the trick. I tried a 12b model and it worked. Now pulling a 70b model to try.
Strangely I had to re-register my admin account HOWEVER, all the previously downloaded models were already there

You had to re-register your admin because the volume where open-webui stores its runtime config was deleted.

Be aware that a 70b model will not fit on your GPU and ollama will load most of it in RAM and use CPU for inference, so it will run pretty slow.

Be aware that a 70b model will not fit on your GPU and ollama will load most of it in RAM and use both GPU and CPU for inference, so it will run pretty slow.

the 70b runs (slow) on my Mac Studio.. what would I need to run the large models? In terms of hardware?

ollama downloads chunks so if you restart the download it should start where the previous download stopped.  Make sure you have plenty of disk available in /var/lib/docker.
The size of the model is shown on the ollama model library, eg llama3.1:70b is 40G so you would need 3 RTX 4060 Ti cards or 2 A10s or one A40.

@nicholhai Can you please close the issue if it's resolved?

Will do. Thank you for all the assistance.

ollama 500, Llama3.1:70b for me it was lack of memory expected 28.8gb actual 8.8gb
Sadge I'll try quantization


Updated release notes file name

Description
Update file name
Issues Resolved
Resolves #[Issue number to be closed when this PR is merged]
Check List

[ ] New functionality includes testing.
[ ] New functionality has a documentation issue. Please link to it in this PR.

[ ] New functionality has javadoc added

[x] Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

There are a lot of build failures. Is this expected?
> Task :e2e-test:log:createDataPrepperNetwork
Creating network 'data_prepper_network'.
Created network with ID '190c52e1b898bf3899d167d43ea016ddf44a2678a124b1087951ae96aee296f1'.

> Task :e2e-test:log:pullOpenSearchDockerImage
Pulling image 'opensearchproject/opensearch:1.3.8' from [https://index.docker.io/v1/.](https://index.docker.io/v1/)

> Task :e2e-test:log:checkstyleIntegrationTest
Error during callback
com.github.dockerjava.api.exception.InternalServerErrorException: Status 500: {"message":"received unexpected HTTP status: 503 Service Unavailable"}
> Task :e2e-test:log:pullOpenSearchDockerImage FAILED


Feature: Sign In pop up requires a close button/icon

I'm unable to close the sign in / create account pop up.

Hey Can you explain more about this issue?

I've confirmed the sign in/create acount form is in its own page and not a modal, sorry.


Create test-coverage.yaml

We should add a badge in the readme next to the tests to see the coverage percentage: