-
Notifications
You must be signed in to change notification settings - Fork 111
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
new: Added jina clip text embedding #408
base: main
Are you sure you want to change the base?
Conversation
new: added resize2square
86287ef
to
82e2d4b
Compare
def resize2square( | ||
image: Image.Image, | ||
size: int, | ||
fill_color: Optional[Union[str, int, tuple[int, ...]]] = None, | ||
resample: Union[Image.Resampling, int] = Image.Resampling.BICUBIC, | ||
) -> Image.Image: | ||
resized_image = resize(image=image, size=size, resample=resample) | ||
|
||
new_image = Image.new(mode="RGB", size=(size, size), color=fill_color) | ||
left = (size - resized_image.size[0]) // 2 | ||
top = (size - resized_image.size[1]) // 2 | ||
new_image.paste(resized_image, (left, top)) | ||
return new_image |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we already have resize
, let's not introduce new functions, we can add an additional parameter like preserve_aspect_ratio
or something like this to resize
.
If it's false then we should just resize image to the required size (preserving aspect ratio is useful when later we have crop)
if config.get("do_normalize", False) or ("mean" in config and "std" in config): | ||
transforms.append( | ||
Normalize( | ||
mean=config.get("image_mean", config.get("mean")), | ||
std=config.get("image_std", config.get("std")), | ||
) | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if config.get("do_normalize", False) or ("mean" in config and "std" in config): | |
transforms.append( | |
Normalize( | |
mean=config.get("image_mean", config.get("mean")), | |
std=config.get("image_std", config.get("std")), | |
) | |
) | |
if config.get("do_normalize", False): | |
transforms.append(Normalize(mean=config["image_mean"], std=config["image_std"])) | |
elif "mean" in config and "std" in config: | |
transforms.append(Normalize(mean=config["mean"], std=config["std"])) |
Adding jinaai/jina-clip-v1
They provided two examples, the first one works and the second one complains about missing
jinaai/jina-clip-v1/sentence_xlnet_config.json
. The output of the first one seems to have small numbers, like they are normallized but its not mentioned so not sure tbh.Update:
The text model needs pooling and normalizing
The image model needs the image to be square
All Submissions:
New Feature Submissions:
pre-commit
withpip3 install pre-commit
and set up hooks withpre-commit install
?New models submission: